Nemotron Streaming

Two NVIDIA streaming ASR models share the NemotronStreamingASR Swift target. Both are 600M-parameter cache-aware FastConformer encoders paired with an RNN-T decoder, both emit native punctuation and capitalization as regular BPE tokens, both run on the Apple Neural Engine via CoreML, and the multilingual variant additionally has MLX bundles for GPU-resident inference. Pick the one that matches your application:

Shared properties

• Native punctuation and capitalization — no post-processor needed; periods, commas, and case are part of the vocabulary
• 600M parameters — cache-aware 24-layer FastConformer encoder + 2-layer LSTM RNN-T decoder + joint network
• Cache-aware streaming — attention and conv caches flow chunk-to-chunk for continuous context across chunk boundaries
• Multiple chunk sizes — multilingual: 80, 320, 560, 1120 ms (320 ms default); English: 80, 160, 560, 1120 ms (160 ms default)
• Multilingual-only: a prompt kernel (Linear 1152→2048→1024) folds a one-hot language slot into every encoded frame, so the same weights serve all 40 language-locales. The English-only variant has no prompt kernel.

Architecture

Three CoreML models pipelined per audio chunk:

No EOU head

Unlike Parakeet-EOU, Nemotron does not emit a dedicated end-of-utterance token. Two ways to segment continuous audio into utterances:

1. External VAD — pair the session with Silero VAD; on sustained silence, call finalize() to commit the current utterance and createSession() for the next one.
2. Punctuation boundary — when the partial transcript ends in ., ?, or !, treat that as a natural commit cue. No extra model, but depends on the audio actually inducing terminal punctuation.

Bundles

Five published variants of Nemotron-3.5-ASR-Streaming-0.6B, plus the older English-only model on the same Swift target:

Upstream: nvidia/nemotron-3.5-asr-streaming-0.6b (multilingual) and nvidia/nemotron-speech-streaming-en-0.6b (English-only).

On the actual cache-aware 320 ms path, MLX 5-bit averages 11.53% WER across six FLEURS languages versus 11.01% for MLX 8-bit. The 5-bit default saves about 193 MB on disk and 192 MB of peak RSS; MLX 4-bit remains a research export because its quality loss is larger.

Quick start — native MLX streaming

The Swift runtime supports the published MLX 5-bit and 8-bit bundles. It defaults to 5-bit, keeps every session's caches independent, and serializes inference when sessions share one loaded model. MLX uses the validated 320 ms geometry and currently does not expose word boosting.

import NemotronStreamingASR

let model = try await NemotronStreamingASRMLXModel.fromPretrained(
    variant: .int5
)
let session = try model.createSession(language: "en-US")

for chunk in microphoneChunks16kHz {
    for partial in try session.pushAudio(chunk) {
        print(partial.text)
    }
}
let final = try session.finalize().last?.text

Quick start — CoreML batch transcription

Conforms to SpeechRecognitionModel, so it drops into any code path that takes a generic STT model:

import NemotronStreamingASR

let model = try await NemotronStreamingASRModel.fromPretrained()
let text = try model.transcribeAudio(audioSamples, sampleRate: 16000, language: "en-US")

Quick start — async streaming

for await partial in model.transcribeStream(audio: samples, sampleRate: 16000) {
    if partial.isFinal { print("FINAL: \(partial.text)") }
    else               { print("... \(partial.text)") }
}

Each PartialTranscript carries text, isFinal (true only for the last partial after finalize()), confidence, and a monotonic segmentIndex.

Long-lived session API (mic input)

let session = try model.createSession()

// each mic chunk:
let partials = try session.pushAudio(float32Chunk16kHz)
for p in partials { showPartial(p.text) }   // isFinal is false mid-stream

// when the utterance ends (VAD silence or explicit stop):
let trailing = try session.finalize()
for p in trailing { commit(p.text) }

CLI

speech transcribe recording.wav --engine nemotron --language en-US                    # batch
speech transcribe recording.wav --engine nemotron --language en-US --stream           # streaming final
speech transcribe recording.wav --engine nemotron --language ja-JP --stream --partial # partials, Japanese
speech transcribe meeting.wav   --engine nemotron --language de-DE                    # any of the 40 locales

Nemotron vs Parakeet-EOU