音源分离

Open-Unmix HQ 将一条立体声音乐轨分离为四个独立声轨 — 人声、鼓、贝斯和其他。四个独立的 BiLSTM 模型（每个声轨一个）在混音 STFT 上生成幅度掩码；可选的维纳后滤波器用于协调它们。通过 MLX 在 Apple Silicon 上运行。

概览

4 stems per track — vocals, drums, bass, other. Each is a 2-channel 44.1 kHz stem file.
Magnitude-mask model — each stem model predicts a non-negative mask applied to the mixture spectrogram; phase is taken from the mixture.
Wiener post-filter (optional) — soft-mask refinement across all 4 stems so they sum coherently back to the mixture. Adds ~0.5 dB SDR.
Small footprint — 8.9M params per stem, ~136 MB total for all 4 stems.
Apache-2.0 — upstream weights under MIT, our CoreML/MLX conversion under Apache-2.0.

架构

Four independent stems, each a copy of the same network:

Stage	Shape / operation
STFT	4096-point FFT, 1024-hop, periodic Hann window, reflect-pad. 2049 frequency bins per frame.
Input normalize	Crop to 1487 bins (≈16 kHz), apply learned per-bin mean + scale from training.
Encoder	Linear 2974 → 512 + BatchNorm + tanh. Input is 2 channels × 1487 bins.
BiLSTM	3 layers, 256 hidden per direction (512 effective). Captures temporal context across frames.
Decoder	Skip-concat of encoder and LSTM outputs (1024) → Linear 1024 → 512 + BN + ReLU → Linear 512 → 4098.
Output denorm + mask	Element-wise multiply with mixture magnitude; phase from mixture; iSTFT overlap-add.
Wiener (optional)	Power-ratio masks across all 4 stem estimates. Refines phase so stems sum to mixture.

模型

Component	Value
Parameters / stem	8.9M
Parameters total (4 stems)	~35.6M
Sample rate	44.1 kHz stereo
Chunk latency	Offline (full-track STFT)
Weights	aufklarer/OpenUnmix-HQ-MLX (safetensors, ~136 MB)
Upstream	sigsep/open-unmix-pytorch (Stöter et al., JOSS 2019)

快速开始 — Swift

import SourceSeparation
import AudioCommon

let separator = try await SourceSeparator.fromPretrained()

let stereo = try AudioFileLoader.loadStereo(
    url: URL(fileURLWithPath: "song.wav"),
    targetSampleRate: 44100
)

let stems = separator.separate(audio: stereo, sampleRate: 44100)
// stems[.vocals], stems[.drums], stems[.bass], stems[.other]
// Each is [[Float]] — left channel, right channel.

try WAVWriter.writeStereo(
    left: stems[.vocals]![0],
    right: stems[.vocals]![1],
    sampleRate: 44100,
    to: URL(fileURLWithPath: "vocals.wav")
)

Pass wiener: true (default) for best quality. Pass targets: [.vocals] to extract only a subset of stems and skip the other models.

命令行

speech separate song.wav                              # all 4 stems into song_stems/
speech separate song.wav --stems vocals               # vocals only
speech separate song.wav --stems vocals,drums         # subset
speech separate song.wav --output-dir /tmp/stems/     # custom output dir
speech separate song.wav --verbose                    # show timing

何时使用

Open-Unmix 适用于…

…当你需要在 Apple Silicon 应用或流水线中进行轻量级离线音源分离时。每个声轨 890 万参数，保持下载和内存开销适中。对大多数流行/摇滚内容，幅度掩码加维纳能产出优质声轨。若需工作室级人声分离的最先进性能，可选用更大的 Demucs/MDX-Net 级模型；此包专注于可嵌入应用的实用端。