Speech-to-Speech Models

Soniqo supports two native MLX model families for conversational speech-to-speech: PersonaPlex 7B and VoiceChat 11B. Both consume speech and generate model audio directly, but they use different duplex architectures and runtime contracts.

Choose a model

These are conversational speech-to-speech models. Hibiki also maps speech directly to speech, but its task is streaming speech translation, so it is documented separately.

PersonaPlex 7B

Full-duplex speech-to-speech dialogue model based on the Moshi architecture (Kyutai). PersonaPlex 7B generates spoken responses directly from spoken input — no intermediate text pipeline required. The model ships with 18 voice presets and is available in 8-bit (recommended) and 4-bit quantization. 8-bit is the default — it is 30% faster and produces coherent responses, while 4-bit degrades output quality.

VoiceChat 11B

Duplex speech-to-speech reference: SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model (Interspeech 2025), cited by NVIDIA's upstream VoiceChat model card.

M5 Pro, 48 GB, Release, 80 ms live cadence, protected-head INT5: three controlled runs measured whole-pipeline RTF 0.94, 0.92, and 0.92, with about 8.70 GB peak RSS. First spoken text arrived in 44.3–46.7 ms and first playable audio in 74.0–76.4 ms. Those are hot compute figures, not learned turn-taking latency; optimized INT8 performance has not yet been remeasured.

VoiceChat is the other native MLX speech-to-speech runtime in speech-swift. It combines causal FastConformer perception, Nemotron-H duplex text/function channels, standalone text-conditioned EAR-TTS, and a 22.05 kHz neural codec. A session accepts 16 kHz mono audio continuously and returns text events plus one 1,764-sample output frame every 80 ms.

Use VoiceChatModel.load(from:), start a VoiceChatSession, and feed audio with pushAudio. A complete bundle must contain encoder/, llm/, and tts/; older understanding-only bundles are rejected.

PersonaPlex architecture and usage

PersonaPlex is a multi-stream autoregressive model with three core components:

The model processes 17 streams simultaneously: 1 text stream + 8 user audio streams + 8 agent audio streams. This architecture enables full-duplex conversation where the model can listen and speak at the same time.

Voice Presets

PersonaPlex includes 18 built-in voice presets across natural and varied styles:

Inner Monologue

PersonaPlex generates two parallel streams at every step: 8 audio codebook tokens for the Mimi codec and one text token for the model’s internal monologue. The text stream is what the model is “thinking” as it speaks — it can diverge slightly from the final audio, but in practice it mirrors the spoken response closely enough to use as a live transcript.

The text tokens come back as raw SentencePiece piece IDs. Decode them with the SentencePieceDecoder that PersonaPlex ships:

import PersonaPlex
import AudioCommon

let model = try await PersonaPlexModel.fromPretrained()
let decoder = model.tokenizer  // SentencePieceDecoder?

let result = model.respond(userAudio: userSamples, voice: .NATM0)
let transcript = decoder?.decode(result.textTokens) ?? ""
print(transcript)        // "Sure, I can help with that..."
playAudio(result.audio)  // 24 kHz mono Float32

In streaming mode, respondStream emits textTokens chunks as they are produced — decode them incrementally to drive a live caption view while the audio is still generating. The --transcript CLI flag does exactly this behind the scenes.

Why it matters: SentencePieceDecoder is built on the shared AudioCommon.SentencePieceModel protobuf reader, so PersonaPlex, OmnilingualASR and any future SentencePiece-based model decode through the same tokenizer implementation. See the SentencePieceModel reference.

System Prompts

Pass any custom system prompt as a plain string — no external tokenization needed:

let response = model.respond(
    userAudio: audio,
    voice: .NATM0,
    systemPrompt: "You enjoy having a good conversation."
)

Or use a built-in preset:

• assistant — General-purpose helpful assistant (default)
• focused — Concise, direct responses
• customer-service — Polite, solution-oriented support agent
• teacher — Patient, explanatory teaching style

CLI Usage

Generate a spoken response from an audio input:

# Basic speech-to-speech
.build/release/speech respond --input question.wav

# Choose a voice preset
.build/release/speech respond --input question.wav --voice NATM0

# Stream audio output during generation
.build/release/speech respond --input question.wav --stream

# Custom system prompt text
.build/release/speech respond --input question.wav --system-prompt-text "You enjoy having a good conversation."

# Use a preset system prompt
.build/release/speech respond --input question.wav --system-prompt customer-service

# Get transcript alongside audio
.build/release/speech respond --input question.wav --transcript

# JSON output with metadata
.build/release/speech respond --input question.wav --json

Options

Sampling parameters can also be overridden:

Streaming

The --stream flag enables real-time audio output. Audio chunks are emitted as they are generated, so playback can begin before the full response is complete. This is particularly useful for interactive applications where low latency matters.

Performance

Model Variants

Swift API

import PersonaPlex
import AudioCommon

let model = try await PersonaPlexModel.fromPretrained()
let response = model.respond(
    userAudio: userSamples,
    voice: .NATM0,
    systemPrompt: "You are a helpful assistant."
)
try WAVWriter.write(samples: response.audio, sampleRate: 24000, to: URL(filePath: "answer.wav"))