Speech Enhancement — DeepFilterNet3
Remove background noise from speech recordings using DeepFilterNet3. The model runs on the Neural Engine via CoreML for efficient inference, while all signal processing (STFT, ERB filterbank, deep filtering) runs on the CPU via Accelerate/vDSP.
The DSP path follows libdf end to end: 1/960 analysis scaling, normalization alpha 0.99, zero-padded deep-filter taps, and correct NumPy C/Fortran matrix order. On a fixed 20-clip VoiceBank-DEMAND subset, Swift scored PESQ 3.097, STOI 0.964, and SI-SDR 19.12 dB versus 3.029, 0.962, and 18.31 dB for official Python DF3.
Before/after sample
A noisy speech clip processed with DeepFilterNet3 so you can hear the CoreML denoiser before running it locally.
Architecture
DeepFilterNet3 uses a dual-decoder architecture that separates spectral envelope enhancement from fine-grained spectral detail recovery.
| Stage | Details |
|---|---|
| STFT | Short-time Fourier transform via vDSP |
| Encoder | 4 SepConv2d layers + SqueezedGRU |
| ERB Decoder | Sigmoid mask applied to ERB-scale frequency bands |
| DF Decoder | 5-tap complex-valued filtering coefficients |
| iSTFT | Inverse STFT to reconstruct the time-domain signal |
The ERB Decoder estimates a gain mask on the Equivalent Rectangular Bandwidth (ERB) scale, handling broad spectral shaping. The DF Decoder predicts 5-tap complex filtering coefficients for fine detail, applying learned filters directly in the frequency domain.
Processing Pipeline
- STFT — Decompose the noisy audio into time-frequency representation using vDSP
- ERB Features — Map STFT bins to ERB-scale frequency bands
- Neural Network — Encoder processes features on Neural Engine; ERB and DF decoders predict enhancement parameters
- ERB Masking — Apply sigmoid gain mask to suppress noise in the spectral envelope
- Deep Filtering — Apply 5-tap complex coefficients for fine spectral detail recovery
- iSTFT — Reconstruct clean audio from the enhanced spectrum
Model
| Variant | Size | Precision |
|---|---|---|
| Default | ~2.2 MB | 8-bit palettized weights, FP16 compute |
CLI Usage
# Denoise audio (output to _denoised.wav)
.build/release/speech denoise noisy.wav
# Specify output file
.build/release/speech denoise noisy.wav -o clean.wav
# Use another compatible HuggingFace repository
.build/release/speech denoise noisy.wav --model owner/DeepFilterNet3-CoreML Options
| Option | Description |
|---|---|
--output, -o | Output file path (defaults to <input>_denoised.wav) |
--model | Compatible HuggingFace model repository ID (default: aufklarer/DeepFilterNet3-CoreML) |
DeepFilterNet3 runs on the Neural Engine via CoreML, not on the GPU via MLX. This means it works efficiently even while other GPU-based models (ASR, TTS) are running. No metallib compilation is required.
Model Downloads
| Model | Size | HuggingFace |
|---|---|---|
| DeepFilterNet3 (CoreML, 8-bit palettized) | ~2.2 MB | aufklarer/DeepFilterNet3-CoreML |
Combining with Other Models
Speech enhancement is particularly useful as a preprocessing step before other models:
- Before transcription — Denoise audio before running ASR to improve word error rate on noisy recordings
- Before speaker embedding — Cleaner audio produces more reliable speaker embeddings
- Before diarization — Noise removal can improve segmentation accuracy
# Denoise then transcribe
.build/release/speech denoise noisy.wav -o clean.wav
.build/release/speech transcribe clean.wavSwift API
import SpeechEnhancement
let model = try await SpeechEnhancer.fromPretrained()
let cleanAudio = try model.enhance(audio: noisySamples, sampleRate: 48000)Also available on Android, and on Linux & Windows through Speech Core (ONNX Runtime).