A small model's prefill activations can price output length too: reasoning-model routing saves 9% to 33% on eight of nine datasets
When routing reasoning models, how many tokens each query will produce is unknown in advance, and most routers price candidates at a fixed or median length. One approach reads the prompt's prefill activations from a frozen small model to predict both each candidate's success probability and its output length; on eight of nine datasets, pricing by predicted length spends 9% to 33% less than median-length pricing at matched accuracy, up to 69% of what perfect per-query cost knowledge would save.