Chatterbox Multilingual

This first-party Soniqo page documents Chatterbox Multilingual from the local speech-swift / speech-core implementation. Hugging Face bundles are linked below after the integration notes.

At a Glance

Use

The snippet below mirrors the current speech-swift API or command exposed by the repo.

import ChatterboxTTS

let model = try await ChatterboxTTSModel.fromPretrained()
let pcm = try model.clone(
    referenceSamples: reference,
    sampleRate: 24_000,
    text: "The cloned voice now speaks a new sentence.",
    languageId: "en"
)

Model Links

• MLX fp16 bundle
• LiteRT bundle
• speech-swift docs
• speech-core header

Implementation Notes

• Download is split between the main Chatterbox bundle and the separate S3 tokenizer repository.
• Pipeline is T3 text-to-speech tokens, S3Gen flow mel decoder, then HiFi-GAN / HiFTGenerator vocoder.
• Reference clips are resampled to 24 kHz for S3Gen and 16 kHz for speaker/token conditioning.
• The MLX clone path and standalone S3Gen synthesis use a balanced memory policy by default: a temporary up-to-512 MB MLX cache cap that does not raise a stricter caller cap, plus cache clears between T3, S3Gen flow, and vocoder stages. Pass memoryOptions: .unrestricted for MLX's default cache behavior.
• Flash Core ML cloning uses MLX reference-audio encoders to prepare conditioning, then runs the exported T3/S3Gen Core ML graphs. CFG stays off by default unless the bundle includes t3/NullTextPrefill.mlmodelc; the public 192-token audio export caps Flash reference prompts at 6 seconds so generated speech has room.
• speech-core's LiteRT path ships a default voice; fp16 T3 is recommended for Arabic quality.