Qwen3-TTS

Qwen3-TTS is a 12Hz codec language model with a Mimi decoder for high-quality text-to-speech synthesis. The model is 4-bit quantized and runs faster than real-time on Apple Silicon.

Pipeline

Speech synthesis follows a three-stage pipeline:

1. Talker — 28-layer transformer that converts input text into first codebook tokens at 12.5 Hz
2. Code Predictor — 5-layer transformer that predicts the remaining 15 codebooks from the first codebook hidden states
3. Mimi Codec Decoder — Converts all 16 codebook tokens into a 24 kHz audio waveform

Architecture

Talker

The Talker is the core autoregressive model that generates codec tokens from text input.

Code Predictor

A lightweight 5-layer transformer that takes the hidden states from the first codebook and predicts the remaining 15 codebooks in parallel. This avoids running the full Talker 16 times per step.

Mimi Codec Decoder

The Mimi decoder converts quantized codec tokens back into audio:

1. RVQ decode (16 codebooks)
2. Pre-convolution (512 to 1024 channels)
3. Pre-transformer (1024 to 512 bottleneck, 8 layers, SwiGLU + LayerScale)
4. Upsample (2x, 2x)
5. SEANet decoder (8x, 5x, 4x, 3x upsample stages)
6. 24 kHz waveform output

Model Variants

CLI Usage

Generate speech from text:

.build/release/audio speak "Hello, world!" --output hello.wav

Options

Examples

# Basic synthesis
.build/release/audio speak "The quick brown fox." -o fox.wav

# Streaming output
.build/release/audio speak "Long passage of text..." --stream -o stream.wav

# Batch synthesis from file
.build/release/audio speak --batch-file sentences.txt --batch-size 4 -o output_dir/

Streaming

The --stream flag enables chunked audio output during generation. Instead of waiting for the entire utterance to complete, audio is emitted in chunks as tokens are produced. Use --first-chunk-frames and --chunk-frames to control the size of each chunk.

Batch Mode

For synthesizing multiple utterances, use --batch-file with a text file containing one line per utterance. The --batch-size flag controls how many utterances are processed in parallel.

Performance

On an M2 Max, Qwen3-TTS achieves an RTF (real-time factor) of approximately 0.55, meaning it generates speech faster than real-time. With compile() warmup, each step takes about 37 ms.

Languages

Qwen3-TTS supports multilingual text-to-speech synthesis. The model automatically detects the input language and generates speech accordingly.

Swift API

import Qwen3TTS

let model = try await Qwen3TTSModel.loadFromHub()
let audio = try await model.speak("Hello, world!")
try audio.write(to: "hello.wav")