A 2B decision model returns one token in 20 ms: joey-1 costs a tenth of mammoth-1 per million tokens
Synopsis
Base Labs released joey-1, a 2B decision model that answers label-style calls such as routing a ticket or flagging a message by returning a single token and a calibrated probability in one forward pass, instead of generating text. At production scale it serves the average 2,000-token request at a P50 end-to-end latency of 20 ms, about 2.5x faster than elephant-1 at 49 ms, and at $0.01 per million tokens it costs a tenth of mammoth-1.
Interpretation
Decision calls get a dedicated model shape: one forward pass returns a label and a confidence score for that label, rather than text that must be read for an answer. LLMs have answered questions such as where a ticket should go, whether a message should be flagged, or whether a request should be approved by generating text, which adds cost and latency to every call and compounds along a chain of decisions. At production scale joey-1 serves the average 2,000-token request at a P50 end-to-end latency of 20 ms, about 2.5x faster than elephant-1 at 49 ms, and its $0.01 per million tokens is a tenth of mammoth-1.
joey-1 is post-trained from Qwen3.5-2B on the same data mixture and recipe as elephant-1 and mammoth-1, and all three share one API, so moving between them is a one-line change. The family now spans three sizes rather than three separate choices: mammoth-1 for decisions where accuracy matters most, elephant-1 as the balanced default, and joey-1 for high-volume, latency-sensitive calls. In Base Labs internal testing, whose details are not yet published, joey-1 is the strongest decision model with 3B or fewer parameters; at release all three accept text and images.
joey-1 is live on Baseten Model APIs with zero data retention by default, and its weights are open on Hugging Face. Model APIs support a 16k context window optimized for low latency, while dedicated deployments unlock the full 262k context window and support fine-tuned weights. Classifying 10 million support tickets a month at about 500 tokens each is 5 billion tokens, roughly $50 a month on joey-1 versus $500 on mammoth-1.
Perspective
The demonstrated scope is the release of the Base Labs decision model family on Baseten Model APIs: joey-1's 20 ms P50 end-to-end latency and $0.01 per million tokens apply to the average 2,000-token request at production scale, and to a scenario of classifying 10 million support tickets a month at about 500 tokens each.
The judgment that joey-1 is the strongest decision model with 3B or fewer parameters comes from Base Labs internal testing, whose details are not yet published; the release also says the research and post-training teams will keep improving the models and expanding the family.