Skip to main content
Back to timeline
arXivSource publication:

Routing stops ranking answers by one score: security queries get their own quality, closing nearly 33 points of the security gap at the same models and budgets

Synopsis

SCRouter splits routing quality into utility for ordinary queries and security for security-sensitive ones, weights the two per query, and recovers 60.9% of the gap between random selection and the per-query oracle on its own benchmark, 12.9 points above the previous best router R2-Router.

AI-generated editorial illustration: Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection

Interpretation

The quality standard for large language model routing is no longer one ruler: it splits by query into utility and security, and lets the two objectives prefer different models and output budgets. Earlier routers ranked model-budget actions with a single quality metric, and R2-Router, which extended the action space to model-budget pairs, still used one quality score. On SCRouterBench, SCRouter-L recovers 60.9% of the gap between random selection and the per-query oracle against 48.0% for R2-Router, a 12.9-point difference, and the two differ by 32.9 points on the security subset.

Routing first decides whether a query is security-sensitive, then predicts utility and security separately for each model-budget pair, and finally trades the weighted score against cost. The construction uses a frozen text encoder with a logistic-regression probe to produce a security weight, a shared budget model to estimate the probability of finishing within budget and the expected output length, and two quality predictors that use that completion probability to estimate utility and security under completed and truncated responses. The benchmark holds 38,869 queries covering utility tasks and nine security dimensions, each answered under all 144 combinations of 12 models and 12 budgets, plus 2,753 out-of-domain queries; the probe classifies 99.6% of security queries correctly in-domain and 74.2% on unseen sources.

The security gain comes mainly from the budget dimension, not from a larger action space alone. Model-only baselines cannot go below the cost of the cheapest model at its default budget, while a safe answer to a harmful request is often a short refusal and security peaks at only 120 tokens. CARROT-KNN chooses among only 12 models and reaches 38.8 on security, close to the 39.1 of R2-Router with all 144 actions, while SCRouter uses the same 144 actions and reaches 72.0.

Perspective

Evaluation is limited to single-turn text queries and a pool of 12 open-weight models, with cost computed from actual token consumption and a fixed per-model price table shared by all routers. The security queries and corresponding model responses contain toxic, biased and harmful content, including jailbreak and prompt-injection attacks, used solely to evaluate routing and released only for research purposes. Per-model security results could reveal which configurations are easiest to attack. SCRouter is a cost-aware routing layer, not a safety guarantee. Multi-turn and multimodal workloads are left for future evaluation.

Out-of-domain performance is lower than in-domain: SCRouter-L drops from 60.9 to 48.1 overall, mainly because its security score falls from 72.0 to 47.3, and the probe classifies only 74.2% of security queries correctly on unseen sources, so about a quarter are routed like utility queries. Cost comparisons use a fixed price table and do not account for time-varying prices or load-dependent serving costs.