Hugging Face turns decisions into an interface: models return option probabilities instead of generating text
Synopsis
Hugging Face has defined a common interface for decision models: given a state and a set of typed questions, a model returns option probabilities, an ordered score or a true/false probability without generating text token by token. Previously such structured judgments relied on prompting a large language model for JSON, on fixed-label classifiers, or on zero-shot classification. Transformers adds DecisionPipeline and related classes, llama.cpp serves the same /v1/systemone endpoint, and compatible autoregressive models can score answer options without producing text.
Interpretation
Decision models separate making a judgment from generating text, so an application receives predefined answers and their probabilities rather than a reply that still has to be parsed. The common prior approach prompts a large language model to return JSON, or uses a conventional classifier whose output classes are baked into its head; zero-shot classification allows labels at inference time but handles only a single categorization question. On one customer-support ticket, the decision interface returns billing at 0.998, technical at 0.001 and sales at 0.001 for the department question, alongside an urgency score and a refund probability, all in one request.
A request has two parts, a state and questions, and questions come in three primitives: choosing among options, scoring on an ordered scale, and answering a yes/no question. Choice computes a logit per option and normalizes with softmax; score returns the expectation of the level distribution, so probabilities of 0.01, 0.26 and 0.72 for low, medium and high give 1.7; the yes/no primitive returns the probability that the answer is true. Transformers supports the interface through DecisionPipeline, AutoModelForDecision and DecisionConfig, llama.cpp serves GGUF decision models on the same /v1/systemone endpoint, and the Hub adds a dedicated decision-model tag for discovery.
Perspective
The report covers the interface specification and tooling: the Transformers decision pipeline, the matching llama.cpp endpoint, the Hub tag, and an example of fine-tuning a decision model with Trainer; the customer-support ticket and its probabilities only illustrate the response format.
Decision models are described as a young and fast-moving space in which each model works differently under the hood; first-class training support in TRL is still an open pull request, and the recipe for turning a generative model into a decision model comes from Unsloth.