Hugging Face makes decisions a model output: no token-by-token text, just option probabilities
Synopsis
Hugging Face has fixed a common interface for decision models: given a state and several typed questions, the model returns option probabilities, an ordered score or a yes/no probability without generating any text. Previously such structured judgments relied on an LLM emitting JSON, sometimes backed by constrained decoding, or on fixed-label and zero-shot classifiers. Transformers, llama.cpp and a Hub tag now follow the same contract, and compatible autoregressive language models can use the interface without retraining.
Interpretation
Decision models make choosing among predefined answers a native model output rather than something parsed out of generated text. Applications have prompted LLMs to return JSON, or used fixed-label classifiers and zero-shot classification; the former does not guarantee the schema, and the latter supplies labels at inference time but covers only a single classification question. On one customer-support ticket, the decision interface returns billing at 0.998, technical at 0.001 and sales at 0.001 for the department question, alongside an urgency score and a refund probability, with zero output tokens.
The interface has two parts, a state and questions, and questions come in three primitives: choice among options, score on an ordered scale, and noul for a yes/no probability. Choice computes one logit per option and normalizes with softmax; score does not take the most likely level but the expectation over level indices, so a distribution of 0.01, 0.26 and 0.72 yields 1.7; noul returns the Bernoulli parameter itself. Transformers adds DecisionPipeline, AutoModelForDecision and DecisionConfig plus a /v1/systemone endpoint; llama.cpp serves GGUF decision models through the same endpoint, and the Hub adds a dedicated tag for models built on different backbones.
Perspective
The demonstrated scope is the three decision primitives in a customer-support routing example, plus the interface's adoption in Transformers, llama.cpp and a Hub tag; it addresses developers building applications that make structured judgments among predefined answers.
Decision models are described as a young and fast-moving space in which each model decides differently under the hood; first-class training support in TRL is still an open pull request, and the fine-tuning example is explicitly called a small toy example.