--- card_version: 1 updated: 2026-09-27 name: GLiNER2.5-Decide for speech-swift status: development-preview # not in a speech-swift release; API and CLI may change upstream_model: fastino/GLiNER2.5-Decide upstream_revision: 7ee5da4c2415e32259bcdc0b1a7367c32ce8d6f6 license: apache-2.0 # upstream weights and GLiNER2 code port_reference_license: mit # gliner2-mlx, Andrew Chen Wang runtime: MLX Swift (speech-swift module "GLiNER") platforms_tested: [macOS on Apple Silicon (M5 Pro)] language: en tasks: [single-label-classification, entity-span-extraction] max_input_tokens: 512 network_at_inference: false parameters_in_export: 486444053 # upstream advertises 340M; this is the tensor count of the exported checkpoint default_variant: int8 variants: # Apple M5 Pro, idle; median ms per request; peak process footprint; max |confidence - upstream| int8: {repo: aufklarer/GLiNER2.5-Decide-340M-MLX-8bit, weights_bytes: 566972210, routing_ms: 7.6, extraction_ms: 8.9, peak_memory_gb: 0.85, max_confidence_drift: 0.006, status: verified} fp16: {repo: aufklarer/GLiNER2.5-Decide-340M-MLX-fp16, weights_bytes: 972940762, routing_ms: 8.8, extraction_ms: 10.0, peak_memory_gb: 1.58, max_confidence_drift: 0.0012, status: verified} fp32: {repo: aufklarer/GLiNER2.5-Decide-340M-MLX, weights_bytes: 1945829412, routing_ms: 11.1, extraction_ms: 12.6, peak_memory_gb: 2.55, max_confidence_drift: 0.0005, status: verified} human_guide: https://soniqo.audio/guides/gliner --- # GLiNER2.5-Decide for speech-swift: model card for agents This card is for coding agents and tools that integrate the model. It covers what the Swift runtime supports, the exact call surface, and which results have been verified. For a human-oriented walkthrough, see . ## What it is GLiNER2.5-Decide is a 340M-class DeBERTa-v3-large encoder from Fastino. It takes text plus a caller-supplied label schema. For classification it returns a probability for every label. For extraction it returns spans of the original text. It does not generate text. speech-swift runs it natively in MLX Swift; no Python is needed at inference time. ## Use it for - Routing a short command or message to one of a fixed set of actions. - Finding mentions such as a person or a time in a short text, with offsets. - Offline or on-device use, where the text should not leave the machine. ## Do not use it for - Multi-label classification, relations, record extraction or joint constraints. They exist upstream but are not in the Swift port, so do not call or emulate them. - Inputs over 512 encoded tokens. They are rejected, never truncated; split the input yourself. - Non-English text. Upstream publishes a separate multilingual model, which this port does not load. - Normalizing dates or times. "six PM" comes back as a text span; converting it is your code's job. - Executing tools or acting on a decision without a policy. Probabilities are model scores, not guarantees. - Negation-sensitive decisions without an extra check. See the known failures below. ## Getting weights MLX conversions of the pinned upstream revision are published on Hugging Face. The runtime downloads them on first use into `~/Library/Caches/qwen3-speech/` and reuses the cache; `offlineMode: true` loads from the cache only. | Variant | Repository | Weights | | --- | --- | ---: | | `int8` (default) | `aufklarer/GLiNER2.5-Decide-340M-MLX-8bit` | 567 MB | | `fp16` | `aufklarer/GLiNER2.5-Decide-340M-MLX-fp16` | 973 MB | | `fp32` | `aufklarer/GLiNER2.5-Decide-340M-MLX` | 1.95 GB | Each bundle has `weights.safetensors`, `config.json`, `encoder_config/config.json`, the tokenizer files and `export.json` (source revision and weight SHA-256). `int8` also has `quantization.json`, which the loader validates. Each repository's card states that variant's measured speed, memory and fidelity. ## Swift API ```swift import GLiNER let model = try await GLiNER.fromPretrained() // variant: .int8 by default // or a local bundle: try await GLiNER.load(from: directoryURL) // [GLiNERChoice] in the order supplied; probabilities sum to 1. let choices = try model.classify( "Remind me to call Dad at six PM.", task: "action", // default labels: ["create_reminder", "send_message", "other"], descriptions: [:] // optional label -> text ) // [label: [GLiNERSpan]]; every requested label is a key, possibly []. let spans = try model.extractEntities( "Remind me to call Dad at six PM.", labels: ["person", "time"], descriptions: ["time": "Time of day or duration mentioned in the command"], threshold: 0.5 // default, 0...1 ) ``` - `GLiNERChoice { label: String, probability: Float }` - `GLiNERSpan { text: String, score: Float, start: Int, end: Int }`: `start` and `end` are **UTF-16** offsets (`NSRange` units), half-open. Upstream Python uses Unicode code points, so offsets differ for text with characters outside the BMP, such as emoji. - Labels: 1 to 255, distinct, not blank. Otherwise `GLiNERError.invalidSchema` is thrown. - Overlapping spans are removed per label, keeping the higher score. - Errors (`GLiNERError`): `invalidConfiguration`, `invalidSchema`, `missingWeight`, `inputTooLong(tokens)`. - Concurrency: use one instance serially. Load once and reuse it; loading takes about 1 s. - `evaluateLayers: true` materializes each encoder layer before building the next. It is meant to lower peak allocations and may be slower; its effect is not yet measured. ## CLI ```sh speech gliner classify "" --labels a,b,c [--variant int8|fp16|fp32] \ [--task action] [--description label=text ...] [--evaluate-layers] [--json] speech gliner extract "" --labels person,time [--variant int8|fp16|fp32] \ [--threshold 0.5] [--description label=text ...] [--evaluate-layers] [--json] ``` - Omit `` to read it from stdin. - `--variant` picks a published bundle (default `int8`); `--model ` overrides its repository; `--model-dir ` loads a local bundle instead. `--model` and `--model-dir` are mutually exclusive. - `--labels` is comma-separated and trimmed. `--description` repeats; split on the first `=`, and the label must appear in `--labels`. - Invalid arguments exit with code 64 before any model loads. Runtime errors, including empty stdin, print `Error: ...` on **stdout** and exit with code 1. Check the exit code before parsing JSON. - Without `--json`, loading messages go to stderr and results to stdout. `--json` output (field names are covered by unit tests): ```json {"text": "Remind me to call Dad at six PM.", "task": "action", "label": "create_reminder", "probability": 0.545, "choices": [{"label": "create_reminder", "probability": 0.545}, {"label": "send_message", "probability": 0.144}, ...], "metrics": {"load_ms": 655.1, "inference_ms": 7.6}} {"text": "Remind me to call Dad at six PM.", "threshold": 0.5, "offset_units": "utf16", "entities": {"person": [{"text": "Dad", "score": 0.995, "start": 18, "end": 21}], "time": [{"text": "six PM", "score": 0.999, "start": 25, "end": 31}]}, "metrics": {"load_ms": 655.1, "inference_ms": 8.9}} ``` Scores and offsets are FP32 library outputs for this sentence (six routing labels: create_reminder, create_calendar_event, send_message, search_notes, set_timer, other). The `metrics` values are the measured INT8 load time and median request times. The top choice at 0.545 shows why callers should gate on a probability threshold rather than acting on the highest label alone. ## Verified results (Apple M5 Pro, idle machine, 2026-09-27) | Variant | Routing (median) | Extraction (median) | Peak process memory | Max confidence drift | | --- | ---: | ---: | ---: | ---: | | `int8` (default) | 7.6 ms | 8.9 ms | 0.85 GB | 0.006 | | `fp16` | 8.8 ms | 10.0 ms | 1.58 GB | 0.0012 | | `fp32` | 11.1 ms | 12.6 ms | 2.55 GB | 0.0005 | - **Fidelity to upstream:** every variant returns the same labels, spans and offsets as the upstream PyTorch model on 24 reference cases; drift is the largest confidence difference. - **Timing:** full request including tokenization, model loaded once, 16 routing cases with six labels and 8 extraction cases with two labels, five timed calls each after five warmups; p95 is within 0.5 ms of the median. Python gliner2-mlx 0.1.2 (FP32) measured 13.7 / 15.0 ms in the same session. - **Memory:** peak physical footprint of the whole process, including the cached relative-position projections. It is not a minimum device requirement. - **Handwritten smoke set:** 12 of 16 routing cases and 7 of 8 extraction cases matched expectations, identically across variants and runtimes. This is not an accuracy benchmark. ## Known failures | Input | Output | Lesson | | --- | --- | --- | | `Do not set a timer.` | `set_timer`, about 0.94 | The model ignores the negation. Gate risky actions on an explicit check or a confirmation step. | | `Call Alice, not Bob, at six PM.` (person) | `Alice` only | The model resolves the recipient rather than listing every mention. Do not rely on it for either behavior. | ## Licensing and attribution - Model weights (`fastino/GLiNER2.5-Decide`) and the GLiNER2 library: Apache-2.0, Fastino. - Swift port: adapted from gliner2-mlx, (c) 2026 Andrew Chen Wang, MIT. The license ships with the module as `Sources/GLiNER/LICENSE-reference`. - The INT8 bundle's quantization follows MLX's affine scheme (Apple Inc., MIT); its license ships in that bundle as `LICENSE-mlx-reference`. ## Links - Upstream model card: - Release post: - GLiNER2: - gliner2-mlx: - speech-swift: