Agents stop routing on the current request alone: session history lifts pass rate 3.3 points at matched offloading
Synopsis
Model routing for multi-turn agents is studied at the user-turn boundary, where a router picks one model for the whole turn before it runs. At a matched offloading rate of roughly a fifth of turns, a router that reads session history passes 28.3% of SheetCopilot tasks against 25.0% for the same router seeing only the current request, a 3.3-point gap, while staying within one point of always using the expensive model. A closed-form cache analysis finds downward switches repay their cache cost in about two calls, while upward switches do not break even under the evaluated conditions.
Interpretation
Model routing inside multi-turn agents is treated as a sequential decision rather than a set of independent requests: which model a turn needs depends on the session and workbook state left by earlier turns. Most routers treat each request as an independent input and judge difficulty from that request alone, but one agent turn can contain many model and tool calls whose difficulty is set by what earlier turns produced. On the multi-turn SheetCopilot version, the history-aware router passes 28.3% of tasks against 25.0% for the same router without history, a 3.3-point gap that is positive for every one of the three candidate pairs.
What the router observes matters more than the decision rule: with session history the router changes which turns it offloads, not how many. The two routers send nearly the same share of turns to the cheap model, 23.3% without history against 22.7% with it on average. The gain is 5.4 points for Sol/Haiku, 2.7 for Sol/Luna and 1.7 for Opus at high versus low reasoning effort, largest for the pair with the widest quality gap.
Switching models is not free, because prompt caching is per model: after a switch the new model cannot reuse the conversation trajectory cached by the previous one. When consecutive turns use the same model, the next turn reuses both the stable system-and-tool prefix and that model's own cached trajectory; after a switch only the fixed system-and-tool prefix stays warm. A closed-form analysis gives the calls needed to break even; at the recorded prices a downward switch to Luna or Haiku repays in about two calls, while an upward switch costs about $4.9 per million tokens of context, ten times the $0.5 downward cost, and never pays back on input under the evaluated conditions.
Perspective
Evaluation covers two multi-turn spreadsheet benchmarks: SheetCopilot contains 221 tasks, of which 112 with multiple subtasks are retained, averaging 2.88 turns and at most seven, while SpreadsheetBench 2 converts 221 end-to-end workflows into conversations averaging 4.76 turns and at most six. Candidate pairs vary one factor at a time: model size within a provider, reasoning effort within one model, and model provider. All policies share one agent runtime, system prompt, tools and execution limits, and costs use prices recorded on August 29, 2026, billing cached input at the cached rate and other input at the uncached rate.
Sessions average nearly three turns with trajectories up to seven, so longer trajectories, independent tasks and cross-sheet interactions remain future work. Decomposing annotated subtasks into sequential turns approximates natural interaction without capturing the full diversity of organic dialogue. The two benchmarks differ in workbook size and domain, so the cost-discount comparison between them is descriptive. The fixed-model baseline is assumed to preserve the growing trajectory cache across every turn of a task, which makes the cost comparison conservative for routed policies.