GLiNER: local decisions and entity extraction
Development preview. The weights are published on Hugging Face; the Swift module is in review and not yet in a speech-swift release.
The GLiNER module runs GLiNER2.5-Decide in MLX Swift. Give it text and a list of allowed labels to obtain classification scores, or ask for entity spans such as a person and time.
How it works
A DeBERTa encoder reads the text and schema together. Classification heads score your labels. Entity heads identify spans of the original sentence. The runtime does not generate a JSON response token by token.
import GLiNER
let model = try await GLiNER.fromPretrained()
let choices = try model.classify(
"Remind me to call Dad at six PM.",
labels: ["create_reminder", "send_message", "other"]
)
let spans = try model.extractEntities(
"Remind me to call Dad at six PM.",
labels: ["person", "time"]
)Weights
Three MLX conversions of the pinned upstream revision 7ee5da4c are published on Hugging Face. The runtime downloads the chosen variant on first use and reuses the local cache afterwards; no Python is involved.
| Variant | Repository | Weights | Routing | Extraction | Peak memory |
|---|---|---|---|---|---|
int8 (default) | aufklarer/GLiNER2.5-Decide-340M-MLX-8bit | 567 MB | 7.6 ms | 8.9 ms | 0.85 GB |
fp16 | aufklarer/GLiNER2.5-Decide-340M-MLX-fp16 | 973 MB | 8.8 ms | 10.0 ms | 1.58 GB |
fp32 | aufklarer/GLiNER2.5-Decide-340M-MLX | 1.95 GB | 11.1 ms | 12.6 ms | 2.55 GB |
Apple M5 Pro, idle machine: median full request including tokenization, and peak process memory. All three variants return the same labels, spans and offsets as the upstream PyTorch model on 24 reference cases, with confidence within 0.006.
Command line
The speech CLI exposes both tasks as speech gliner classify and speech gliner extract. Pass labels as a comma-separated list and repeat --description label=text as needed. Omit the text to read it from stdin. --variant picks int8 (default), fp16 or fp32, and --model-dir loads a local bundle instead of downloading. Invalid arguments are rejected before the model loads. With --json, the output includes every label probability, or spans with UTF-16 offsets, plus load and inference timings.
swift build -c release --product speech --disable-sandbox
scripts/build_mlx_metallib.sh release
.build/release/speech gliner classify "Remind me to call Dad at six PM." \
--labels create_reminder,send_message,set_timer,other
.build/release/speech gliner extract "Remind me to call Dad at six PM." \
--labels person,time --jsonThe model card for agents is a Markdown file that lists the exact API, output fields and verified results.
Accuracy on a small command set
Sixteen handwritten routing examples matched expectations in 12 cases; eight extraction examples matched in 7 cases. The Swift port reproduces the upstream PyTorch model on 24 reference cases: identical token IDs, labels and spans, with scores within 0.001. These small sets check behavior and fidelity; they are not a broad accuracy benchmark.
Scope and limitations
- Single-label classification and entity spans; up to 512 encoded tokens per request.
- Span/count_lstm checkpoints with shared relative attention, in FP32, FP16 or INT8. Boundary models, relation graphs and joint constraints are not part of this initial API.
- Entity offsets use UTF-16, matching NSRange. Dates and times require separate normalization.
- Scores do not guarantee that a decision is correct. No tool is executed by the module.
- Model download size, process memory, and unified-memory physical footprint are different measurements.
Precision and memory
FP16 halves weight storage to 973 MB. INT8 quantizes the encoder matrices and token embeddings to 8 bits in groups of 64, with heads, normalization and activations in FP16, for a 567 MB file. The runtime keeps INT8 matrices packed and decodes only the embedding rows a request uses.
The encoder's relative-position projections depend only on weights, so they are computed once at load instead of on every request. The optional GLiNER.load(from:evaluateLayers:) mode materializes encoder layers one at a time to lower peak allocations.