PISA-E turns ensemble routing into a per-prompt decision, beating the strongest ensemble by 2.09 points when quality comes first
Synopsis
PISA-E decides both which large language models to combine and how many to invoke for each prompt at inference time, reusing PISA's per-model capability estimates to predict ensemble success without training separate estimators for model combinations. Across four routing benchmarks it reaches macro-average task performance 2.09 percentage points above the strongest ensemble competitor in-distribution and 2.87 points out-of-distribution when quality is prioritized, and at selected lower-cost operating points it also cuts average inference cost by roughly 36% and 25%.
Interpretation
A multi-model ensemble no longer keeps a fixed member count; for each prompt it decides which models to call and how many. Earlier ensemble methods either adapted composition while fixing cardinality at training or configuration time, or relied on classification-specific aggregation or sequential execution, so the number of invoked models could not follow the prompt and the cost preference. On the four benchmarks LLMRouterBench Track-2, RouterBench, FusionBench and SPROUT, PISA-E attains the highest macro-average task performance among the evaluated ensemble approaches under both in-distribution and out-of-distribution conditions.
The method first estimates how likely each candidate model is to answer a given prompt correctly, then uses those probabilities to compute the chance that a majority of a model subset is correct. PISA clusters prompt embeddings with X-means, uses soft memberships instead of hard assignments, and trains an independent L2-regularized logistic predictor per model; PISA-E feeds the same capability estimates into a Poisson-binomial dynamic program for the strict-majority success probability, which serves as the ensemble-success surrogate. Candidate subsets are odd-length prefixes of the single-model utility ranking and are compared under an explicit cost-quality preference; the selected models are queried in parallel and their responses combined by capability-weighted BERTScore semantic similarity.
The cost-quality preference can be adjusted at inference time, with one fitted set of capability estimators serving multiple operating points. When quality is prioritized, PISA-E reaches 75.27% macro-average task performance in-distribution and 63.24% out-of-distribution, exceeding the strongest ensemble competitors by 2.09 and 2.87 percentage points respectively. At selected lower-cost operating points it still exceeds the strongest ensemble competitors' macro-average task performance while reducing average inference cost by approximately 36% in-distribution and 25% out-of-distribution.
Perspective
Evaluation covers the four benchmarks LLMRouterBench Track-2, RouterBench, FusionBench and SPROUT; out-of-distribution conditions train only on in-distribution task groups and test on held-out groups, and all methods use the model responses supplied by the benchmarks. The intended reader is a designer of inference services that trade cost against quality over a pool of models.
The strict-majority success probability is only a surrogate for ensemble success, since the final response also depends on relationships among the generated responses; the greedy prefix construction does not guarantee a globally optimal subset. Fixed-cardinality configurations remain competitive on some benchmarks, and a fixed configuration has higher HV on RouterBench.