PersonaPlex

Full-duplex speech-to-speech dialogue model based on the Moshi architecture (Kyutai). PersonaPlex 7B generates spoken responses directly from spoken input — no intermediate text pipeline required. The model ships with 18 voice presets and is available in 8-bit (recommended) and 4-bit quantization. 8-bit is the default — it is 30% faster and produces coherent responses, while 4-bit degrades output quality.

Architecture

PersonaPlex is a multi-stream autoregressive model with three core components:

The model processes 17 streams simultaneously: 1 text stream + 8 user audio streams + 8 agent audio streams. This architecture enables full-duplex conversation where the model can listen and speak at the same time.

Voice Presets

PersonaPlex includes 18 built-in voice presets across natural and varied styles:

Inner Monologue

PersonaPlex generates two parallel streams at every step: 8 audio codebook tokens for the Mimi codec and one text token for the model’s internal monologue. The text stream is what the model is “thinking” as it speaks — it can diverge slightly from the final audio, but in practice it mirrors the spoken response closely enough to use as a live transcript.

The text tokens come back as raw SentencePiece piece IDs. Decode them with the SentencePieceDecoder that PersonaPlex ships:

import PersonaPlex
import AudioCommon

let model = try await PersonaPlexModel.fromPretrained()
let decoder = try model.makeTextDecoder()  // SentencePieceDecoder

let result = model.respondWithTranscript(userAudio: userSamples, voice: .NATM0)
let transcript = decoder.decode(result.textTokens)
print(transcript)        // "Sure, I can help with that..."
playAudio(result.audio)  // 24 kHz mono Float32

In streaming mode, respondStream emits textTokens chunks as they are produced — decode them incrementally to drive a live caption view while the audio is still generating. The --transcript CLI flag does exactly this behind the scenes.

Why it matters: SentencePieceDecoder is built on the shared AudioCommon.SentencePieceModel protobuf reader, so PersonaPlex, OmnilingualASR and any future SentencePiece-based model decode through the same tokenizer implementation. See the SentencePieceModel reference.

System Prompts

Pass any custom system prompt as a plain string — no external tokenization needed:

let response = model.respond(
    userAudio: audio,
    voice: .NATM0,
    systemPrompt: "You enjoy having a good conversation."
)

Or use a built-in preset:

• assistant — General-purpose helpful assistant (default)
• focused — Concise, direct responses
• customer-service — Polite, solution-oriented support agent
• teacher — Patient, explanatory teaching style

CLI Usage

Generate a spoken response from an audio input:

# Basic speech-to-speech
.build/release/audio respond --input question.wav

# Choose a voice preset
.build/release/audio respond --input question.wav --voice NATM0

# Stream audio output during generation
.build/release/audio respond --input question.wav --stream

# Custom system prompt text
.build/release/audio respond --input question.wav --system-prompt-text "You enjoy having a good conversation."

# Use a preset system prompt
.build/release/audio respond --input question.wav --system-prompt customer-service

# Get transcript alongside audio
.build/release/audio respond --input question.wav --transcript

# JSON output with metadata
.build/release/audio respond --input question.wav --json

Options

Sampling parameters can also be overridden:

Streaming

The --stream flag enables real-time audio output. Audio chunks are emitted as they are generated, so playback can begin before the full response is complete. This is particularly useful for interactive applications where low latency matters.

Performance

Model Variants

Swift API

import PersonaPlex

let model = try await PersonaPlexModel.loadFromHub()
let response = try await model.respond(
    audioFile: "question.wav",
    voice: .NATM0,
    systemPrompt: .assistant
)
try response.audio.write(to: "answer.wav")