Qwen3-ASR

Qwen3-ASR is a state-of-the-art multilingual automatic speech recognition model. It runs on-device using Metal GPU acceleration via MLX, with 4-bit, 5-bit, or 8-bit quantization for efficient memory usage. Available in 0.6B and 1.7B parameter variants. 5-bit is the new accuracy/memory sweet spot — see benchmarks.

Pipeline

The Qwen3-ASR inference pipeline processes audio through four stages:

Performance

Apple M5 Pro, 48 GB. LibriSpeech test-clean n=200, isolated per-engine. RTF < 1.0 = faster than real-time.

Model Variants

CLI Usage

Transcribe an audio file with the default Qwen3-ASR model:

.build/release/speech transcribe recording.wav

Options

# Use the larger 1.7B model
.build/release/speech transcribe recording.wav --model 1.7b

# Specify language
.build/release/speech transcribe recording.wav --language en

# Streaming mode with partial results
.build/release/speech transcribe recording.wav --stream --partial

Swift API

Use the Qwen3ASR module to transcribe audio programmatically:

import Qwen3ASR

// Load the model (downloads from HuggingFace on first use)
let model = try await Qwen3ASRModel.fromPretrained()

// Transcribe audio samples (16 kHz mono Float)
let text = model.transcribe(audio: audioSamples, sampleRate: 16000)
print(text)

CoreML Encoder (Neural Engine)

Run the audio encoder on the Neural Engine via CoreML, with the text decoder on GPU via MLX. This hybrid approach lowers power consumption and frees the GPU for concurrent workloads.

import Qwen3ASR

let encoder = try await CoreMLASREncoder.fromPretrained()
let model = try await Qwen3ASRModel.fromPretrained()
let text = try model.transcribe(
    audio: samples, sampleRate: 16000,
    coremlEncoder: encoder
)

# CLI
.build/release/speech transcribe recording.wav --engine qwen3-coreml

INT8 palettized (180 MB, cosine similarity > 0.999) is the default. An INT4 variant (90 MB) is also available for size-constrained deployments.

Streaming Mode

Streaming mode uses VAD (voice activity detection) to segment audio into chunks and transcribe them incrementally. This is useful for long recordings or real-time processing.

# Stream with default segment size
.build/release/speech transcribe recording.wav --stream

# Control maximum segment duration
.build/release/speech transcribe recording.wav --stream --max-segment 15

# Show partial (in-progress) results as they arrive
.build/release/speech transcribe recording.wav --stream --partial

The --max-segment flag controls the maximum chunk duration in seconds. The --partial flag enables partial result output, showing words as they are decoded.

Supported Formats

Qwen3-ASR accepts the following audio formats. All input is automatically resampled to 16 kHz mono internally.

• WAV — uncompressed PCM
• M4A — AAC-encoded audio
• MP3 — MPEG Layer III
• CAF — Apple Core Audio Format

Cooperative cancellation

For MLX transcription, Qwen3ASRModel.transcribeCheckingCancellation(audio:sampleRate:options:) throws CancellationError when task cancellation is observed before feature extraction, encoding, decoder prefill, or a decoding step. GPU work already submitted can finish. The synchronous transcribe and transcribeBatch APIs retain their non-throwing behavior and ignore task cancellation. speech-server uses the cancellation-aware entry point for Qwen3-ASR.

MLX cache budget

Loading Qwen3-ASR bounds the process-wide MLX pool of reusable Metal buffers: min(1 GiB, RAM / 8) for 0.6B and min(4 GiB, RAM / 4) for 1.7B. Other MLX models share this pool. The ceilings apply to cached scratch buffers, not active weights or the total process footprint.

The effective limit is the lowest of the caller’s existing MLX.Memory.cacheLimit and every loaded ASR instance’s ceiling. Instances can unload in any order; the last unload or instance destruction restores the caller’s budget. A tighter limit set by the caller while models are loaded is preserved.

Single-input transcription clears reusable buffers when its decoder returns or throws, including cancellation inside the decoder. Active arrays and model weights remain loaded. The greedy transcribeBatch decoder retains the bounded pool between batches. No API options or CLI flags change.

Clearing the buffer pool lowers retained memory but requires new allocations on later requests, so short requests can take longer. Waiting for submitted decoder work to finish makes cleanup deterministic, including cancellation. Actual latency and process memory depend on the hardware, model, and audio.