p99 and VAP — Turn-taking models

p99 estimates end-of-turn probability from one audio channel. VAP forecasts speech activity from synchronized user and agent channels. The application uses these scores to decide when to answer or interrupt.

Build the SDK from source for these APIs. Swift uses SpeechVAD; C++ requires SPEECH_CORE_WITH_ONNX=ON. The Android API uses ONNX Runtime. These standalone APIs leave the existing pipeline defaults in place.

Models

p99 turn-1-mini5.3M · CoreML · Float3216 kHz · 160 ms · 1 chApache-2.0
VAP (otoSpeech)5.8M · CoreML / ONNX · Float3216 kHz · 20 s · 2 chCC BY 4.0 / MIT

Swift API

p99 buffers incomplete steps. Each 160 ms step returns eight 20 ms frames. Use availableAfterSeconds for decision timing, and reset the session between conversations.

import SpeechVAD

let p99 = try await P99TurnModel.fromPretrained()
let session = try p99.createSession()
let frames = try session.pushAudio(pcm16k)
for frame in frames {
    print(frame.probability, frame.availableAfterSeconds)
}
try session.reset()

let vap = try await VAPModel.fromPretrained()
let forecast = try vap.forecast(userAudio: userPcm, agentAudio: agentPcm)
print(forecast.pNow, forecast.pFuture, forecast.voiceActivity)

VAP uses the latest 20 seconds at 16 kHz. Supply equal-length channels and zeros for an inactive agent. Shorter inputs receive leading silence. Each forecast recomputes the window; choose a request interval that fits the device.

C++

#include <speech_core/models/onnx_vap.h>

speech_core::OnnxVap model("models/vap.onnx");
auto forecast = model.forecast(user.data(), agent.data(), user.size(), 16000);
// p_now[0]: user, p_now[1]: agent
// p_future and voice_activity use the same channel order.

Android

val directory = ModelManager.ensureVapModels(context)
withContext(Dispatchers.Default) {
    VoiceActivityProjection(VoiceActivityProjectionConfig(directory)).use { model ->
        val forecast = model.forecast(userPcm, agentPcm)
        println(forecast.userNearTerm)
        println(forecast.agentNearTerm)
    }
}

Downloads

Benchmark results

TurnBench · Soniqo

Licences