# transformers

Hugging Face has defined a common interface for decision models: given a state and a set of typed questions, a model returns option probabilities, an ordered score or a true/false probability without generating text token by token. Previously such structured judgments relied on prompting a large language model for JSON, on fixed-label classifiers, or on zero-shot classification. Transformers adds DecisionPipeline and related classes, llama.cpp serves the same /v1/systemone endpoint, and compatible autoregressive models can score answer options without producing text.

Hugging Face has defined a common interface for decision models: given a state and a set of typed questions, a model returns option probabilities, an ordered score or a true/false probability without generating text token by token. Previously such structured judgments relied on prompting a large language model for JSON, on fixed-label classifiers, or on zero-shot classification. Transformers adds DecisionPipeline and related classes, llama.cpp serves the same /v1/systemone endpoint, and compatible autoregressive models can score answer options without producing text.

Hugging Face has fixed a common interface for decision models: given a state and several typed questions, the model returns option probabilities, an ordered score or a yes/no probability without generating any text. Previously such structured judgments relied on an LLM emitting JSON, sometimes backed by constrained decoding, or on fixed-label and zero-shot classifiers. Transformers, llama.cpp and a Hub tag now follow the same contract, and compatible autoregressive language models can use the interface without retraining.

Hugging Face has fixed a common interface for decision models: given a state and several typed questions, the model returns option probabilities, an ordered score or a yes/no probability without generating any text. Previously such structured judgments relied on an LLM emitting JSON, sometimes backed by constrained decoding, or on fixed-label and zero-shot classifiers. Transformers, llama.cpp and a Hub tag now follow the same contract, and compatible autoregressive language models can use the interface without retraining.

This Hugging Face post describes how transformers can now load GGUF quantized checkpoints directly through from_pretrained with a gguf_file argument: on Apple Silicon it reuses ggml Metal kernels distributed via the kernels library (packed-quantized weight matmul, fused normalization, flash attention, gated delta net, plus a home-grown topk for MoE routing) and trims generate overhead by dropping the redundant attention mask early and deferring the stopping check asynchronously; on a MacBook Pro M2 Max the authors measure generation speed close to llama.cpp across three checkpoints, report file-size tradeoffs for Q4_K_M, Q5_K_M and Q6_K, and state that packed inference is MPS-only with architecture coverage for Qwen3.5 dense and MoE (including compatible Qwen3.

This Hugging Face post describes how transformers can now load GGUF quantized checkpoints directly through from_pretrained with a gguf_file argument: on Apple Silicon it reuses ggml Metal kernels distributed via the kernels library (packed-quantized weight matmul, fused normalization, flash attention, gated delta net, plus a home-grown topk for MoE routing) and trims generate overhead by dropping the redundant attention mask early and deferring the stopping check asynchronously; on a MacBook Pro M2 Max the authors measure generation speed close to llama.cpp across three checkpoints, report file-size tradeoffs for Q4_K_M, Q5_K_M and Q6_K, and state that packed inference is MPS-only with architecture coverage for Qwen3.5 dense and MoE (including compatible Qwen3.