FlexRouter picks complementary model subsets with determinantal point processes, lifting average Success@10 to 0.8632 and 0.9914 on RouterEval

It changes the routing objective from scoring models independently and taking top-k to maximizing the probability that at least one selected model is correct, i.e., answer coverage. Prior methods score models independently and ignore model correlation, so they often pick redundant models that share failure modes; this work formulates routing as a coverage-oriented subset selection problem and argues this matches practical pipelines where multiple candidates are generated and a verifier or user picks the final answer. The paper defines a coverage reward (a selected subset succeeds if it intersects the correct set) and evaluates with Success@k on RouterEval, explicitly noting that this is a routing-stage coverage metric rather than final deployed accuracy.

It parameterizes the routing policy with a determinantal point process (DPP) so that subset probability captures both model quality and inter-model redundancy. The kernel combines a query-dependent quality score with cosine similarity between model embeddings: diagonal entries capture individual competence, off-diagonal entries penalize jointly selecting correlated models, so the determinant favors high-quality yet complementary subsets. The paper gives the kernel construction and its positive semi-definite guarantee, and in an ablation compares a cosine kernel with an RBF kernel: cosine yields consistently higher diversity with competitive success rates in both settings and is adopted as the default.

It introduces a coverage training objective based on marginalizing over failure sets, requiring no ground-truth target subset. Standard DPP maximum likelihood needs an observed target subset, but here only binary correctness labels per query-model pair exist and no unique optimal subset is defined; the authors instead minimize the negative log probability that a sampled subset lies entirely inside the failure set, adding a binary cross-entropy auxiliary loss on per-model correctness. The paper derives the failure-set marginal probability (Appendix Proposition 3) and reports a supervision ablation: with only 25% of training queries, average Success@10 is 0.860, within 2.4 percentage points of full supervision.

At inference it uses greedy selection by marginal log-determinant gains with an adaptive stopping rule, so subset size follows query difficulty without a predefined budget. Top-k routing enforces a fixed budget, whereas the DPP log-determinant objective is non-monotone: adding a highly correlated model can decrease the subset score, so the stopping rule naturally halts when candidates offer no unique contribution. The paper reports the coverage-cost trade-off across stopping thresholds: at 0.2 the average subset size drops from 10 to 6.41 with a 1.13% coverage drop; at 0.5 the size drops to 1.41 with a 10.57% coverage drop.

The work targets routing-stage candidate-pool construction and fits pipelines that generate multiple candidates in parallel and then let a verifier, reward model, LLM-as-judge reranker, self-consistency aggregation, or human pick the final answer; the authors state that Success@k is a routing-stage coverage metric, not final deployed accuracy. It assumes a large candidate pool and binary correctness labels for each query-model pair, encodes queries with RoBERTa-base into a shared 128-dimensional space, and uses greedy MAP inference with adaptive stopping and a maximum subset size of 10. For serving teams that want to allocate compute by query difficulty, the stopping threshold offers a direct coverage-cost knob.

The paper itself flags open questions: downstream selection is not guaranteed to recover a lone correct answer, so end-to-end evaluation with a concrete selector remains future work; diversity is measured at the model level using fixed embeddings, which does not necessarily imply output-level semantic diversity per query, since RouterEval provides correctness labels but not full generated responses; cost is proxied by the number of invoked models (average subset size), while real deployment cost also depends on model-specific latency, token price, output length, batching, and hardware; and the supervision ablation covers only query-level label sparsity, not model-level label sparsity. In addition, this reading is full text, but some table values appear as images, so the specific numbers cited here follow the textual descriptions.