A small model's prefill activations can price output length too: reasoning-model routing saves 9% to 33% on eight of nine datasets
Synopsis
When routing reasoning models, how many tokens each query will produce is unknown in advance, and most routers price candidates at a fixed or median length. One approach reads the prompt's prefill activations from a frozen small model to predict both each candidate's success probability and its output length; on eight of nine datasets, pricing by predicted length spends 9% to 33% less than median-length pricing at matched accuracy, up to 69% of what perfect per-query cost knowledge would save.
Interpretation
When routing reasoning models, a frozen small model's prefill activations can yield both each candidate's success probability and its output length before any candidate is called, and the predicted length can set the price. Earlier prefill routers predicted only correctness and priced each candidate at the median training output length, effectively assuming every query produces the same number of tokens. On held-out test problems across nine datasets at billed prices, the approach spends significantly less than median-length pricing at matched accuracy on eight datasets, from 9.4% on CodeContests to 33.0% on SuperGPQA.
The saving comes from deciding query by query whether a strong candidate with widely varying length is worth calling, not merely from price gaps between candidates. With every candidate charged the same price per token and actual lengths kept, the cost probe still saves 13.3% (LiveCodeBench), 14.3% (Omni-MATH) and 36.3% (MMLU-Pro) relative to median-length pricing. Removing the cheapest candidate, gpt-oss-20b at low effort, drops the saving from 27.8% to 6.5% on LiveCodeBench and from 31.5% to 16.2% on Omni-MATH.
Output length depends mostly on the problem, so the same questions are long for every model; on math and code data harder problems are longer, and pricing from predicted success alone comes close to the cost probe. On MMLU-Pro and BIG-Bench Extra Hard, length decouples from empirical difficulty, and pricing from predicted success falls 18.4 and 22.6 points behind the cost probe. On MMLU-Pro, pricing from empirical difficulty plus subject saves under 4% relative to median length, against 29% for the cost probe (one training set).
The cost probe keeps most of its saving with few labels, when onboarding a new model, when transferring to new benchmarks, and in mixed traffic. With 20 labelled problems, the probe keeps 51% to 81% of its full-data saving on five datasets, while external estimators save at most 10.3%. On a mixture of all nine benchmarks, a single probe not given the benchmark identity saves 22.4% beyond pricing each benchmark at its own median length, with intervals excluding zero in all ten training sets.
Perspective
The demonstrated setting is five reasoning candidates (two gpt-oss-20b efforts, deepseek-v4-flash, two gpt-oss-120b efforts) evaluated on held-out test problems from nine benchmarks at billed prices, plus three further model families on three datasets. The intended reader is an engineering or research team deploying multi-model routing; the probe needs output-length labels per candidate, a few dozen for most datasets.
The saving needs a cheap candidate that solves a good share of problems, and most of it disappears without gpt-oss-20b at low effort. Telling apart two strong candidates of similar price remains open: billing gpt-oss-120b at high effort at deepseek-v4-flash's token rates, perfect cost knowledge would save 11% to 36% within that pair, while the router saves at most 7.0% relative to calling deepseek-v4-flash alone. Each interval covers one comparison with no correction for multiple comparisons, and test problems are treated as independent.