VoxCPM2

VoxCPM2 is a 2B-parameter, tokenizer-free diffusion-autoregressive TTS model from OpenBMB. It synthesises 48 kHz studio-quality audio in 30 languages with three production modes: zero-shot, single-reference voice cloning, and natural-language voice design ("a young female voice, warm and gentle"). On Apple Silicon it runs natively via MLX in bf16, int8, or int4 — the int8 bundle round-trips through Qwen3-ASR with 0% WER on the 8-sentence test harness and ~RTF 1.0.

What it is

• 48 kHz output — the only on-device engine in this stack with studio sample rate. Every other TTS module maxes at 24 kHz.
• Voice design — natural-language style control: --voxcpm2-instruct "young female voice, warm and gentle". None of the other engines expose this.
• Voice cloning — single-reference cloning from a 16 kHz clip; "ultimate cloning" (reference audio + transcript) for prosody preservation.
• 30 languages — English, Chinese, Indonesian, Japanese, Korean, and more. Auto-detected from text.
• Apache 2.0 — model weights inherit upstream openbmb licence; our Swift port is the same.

Architecture

Five cooperating components produce a 48 kHz waveform:

Bundles

Three quantisation variants, all converted from upstream PyTorch via the openbmb/VoxCPM2 checkpoints. Quantisation applies to the Linear projections inside the LM / residual LM / DiT estimator / projection heads; the AudioVAE vocoder stays at fp16/bf16 because quantising it hurts audio quality.

Round-trip ASR (Qwen3-ASR 0.6B, 8-sentence harness, M-series Apple Silicon):

int8 is the recommended default — it matches the upstream Python pipeline bit-for-bit on the LM path while being faster and 40 % smaller than bf16. int4 is the smallest bundle with acceptable WER for casual use.

Quick start

import VoxCPM2TTS

let tts = try await VoxCPM2TTSModel.fromPretrained()  // defaults to bf16
let audio = try await tts.generate(text: "Hello from VoxCPM2.", language: "english")
// audio: [Float] at 48 kHz mono

Pass an explicit model ID to pick the int8 / int4 bundle:

let tts = try await VoxCPM2TTSModel.fromPretrained(
    modelId: "aufklarer/VoxCPM2-MLX-int8"
)

Voice design (instruction-driven)

Pass a natural-language style description; the model conditions synthesis on it without a reference audio sample:

let audio = try await tts.generateVoxCPM2(
    text: "Welcome to the show.",
    instruct: "A young woman, gentle and warm voice."
)

Voice cloning

Single-reference cloning from a 16 kHz mono clip:

let ref = try AudioFileLoader.load(url: URL(fileURLWithPath: "speaker.wav"),
                                   targetSampleRate: 16000)
let audio = try await tts.generateVoxCPM2(
    text: "This is a cloned voice.",
    refAudio: ref
)

Ultimate cloning — pass both reference audio and the matching transcript so the LM can also condition on the lexical context, preserving prosody and accent more faithfully:

let audio = try await tts.generateVoxCPM2(
    text: "Hello from the cloned voice.",
    refAudio: ref,
    promptText: "this is what the reference clip actually said",
    promptAudio: ref
)

CLI

speech speak "Hello there." \
    --engine voxcpm2 \
    --voxcpm2-variant int8 \
    --output hello.wav

# Voice design
speech speak "Welcome to the show." \
    --engine voxcpm2 \
    --voxcpm2-instruct "A young woman, gentle and warm voice." \
    --output design.wav

# Voice cloning
speech speak "This is a cloned voice." \
    --engine voxcpm2 \
    --voice-sample speaker.wav \
    --output clone.wav

Flags: --voxcpm2-variant {bf16,int8,int4}, --voxcpm2-instruct, --voxcpm2-ref-audio, --voxcpm2-prompt-audio + --voxcpm2-prompt-text, --voxcpm2-cfg-value, --voxcpm2-timesteps, --voxcpm2-max-tokens, --voxcpm2-min-tokens, --seed for reproducible synthesis.

Picking among speech-swift TTS modules

Responsible use

Voice cloning is included. Obtain consent for any voice you clone and don't use the model to impersonate individuals, generate disinformation, or commit fraud. The full safety guidance from openbmb/VoxCPM2 applies.