Whole-Pool Setwise Reranking with Long-Context LLMs: DualEnd Builds a Full 100-Candidate Ranking in 50 Comparisons, 59.4% Fewer Than Windowed HeapSort
Related research and updates
Synopsis
The work introduces Whole-Pool Setwise reranking, in which each comparison ranks the entire candidate pool, and proposes DualEnd, which jointly selects the candidates predicted to be most and least relevant and fills the ranking from both ends, constructing a complete ranking of 100 candidates in 50 LLM comparisons; with nine open-weight LLMs on TREC DL19 and DL20 it requires 59.4% fewer comparisons than top-oriented windowed Setwise with heapsort and 88.8% fewer than with bubblesort, keeps nDCG@100 within 0.008 of a single-end whole-pool top-oriented approach while roughly halving token consumption and ranking time, and across six BEIR datasets reduces mean token consumption and ranking time by 49.4% and 50.8% relative to that single-end approach.
Figure 1: One LLM comparison in (a) windowed Setwise, (b) WP-T and (c) WP-DE. Dashed boxes mark input candidates, blue and orange indicate predicted best and worst candidates. Windowed selections require sorting, Whole-Pool selections fix one or both endpoints through swaps.
arXivInterpretation
Introduces Whole-Pool Setwise reranking: when the entire retrieved candidate pool fits within the context window, each comparison ranks the whole pool rather than performing repeated local comparisons. Prior LLM rerankers produce rankings through repeated local comparisons (listwise, pairwise, or pointwise), requiring many sequential model calls; this work raises the comparison granularity to the whole pool, using long-context capability to compress the number of calls. The abstract states the method definition and experimental setup: nine open-weight LLMs, TREC DL19 and DL20, and six BEIR datasets, with reported numbers for comparisons, nDCG@100, token consumption, and ranking time.
Proposes DualEnd, which in a whole-pool comparison jointly selects the candidates predicted to be most and least relevant, filling the ranking from both ends. Relative to a single-end, top-oriented whole-pool approach, DualEnd extends selection to both ends of the pool, reducing the comparisons needed to build a complete ranking. The abstract reports that DualEnd constructs a complete ranking of 100 candidates in 50 LLM comparisons, with nDCG@100 within 0.008 of the single-end whole-pool top-oriented approach and roughly halved token consumption and ranking time.
On TREC DL19 and DL20, DualEnd uses substantially fewer comparisons than top-oriented windowed Setwise baselines, even though those baselines target only top-10 rankings while DualEnd targets the full ranking. It requires 59.4% fewer comparisons than windowed Setwise with heapsort and 88.8% fewer than with bubblesort, showing that the whole-pool dual-end strategy cuts sequential calls while pursuing a more complete ranking objective. The abstract gives these percentages explicitly and states that the baselines target only top-10 rankings while DualEnd targets the full ranking, directly specifying the comparison conditions.
Across six BEIR datasets, DualEnd reduces mean token consumption by 49.4% and ranking time by 50.8% relative to the single-end whole-pool top-oriented approach. It extends the efficiency result from the TREC collections to multiple BEIR datasets and reports average token and time reductions across them. The abstract reports mean token consumption and ranking time reductions on six BEIR datasets and describes competitive effectiveness across several backbones.
Perspective
The results apply when the entire retrieved candidate pool fits within a long-context window, a premise the method itself depends on; experiments cover TREC DL19 and DL20 plus six BEIR datasets, with nine open-weight LLMs as backbones. For retrieval-system developers who want a complete ranking with fewer sequential LLM calls, DualEnd offers a concrete path of filling the ranking from both ends; for settings that care only about top-10 rankings, the windowed Setwise baselines in the abstract are the closer comparison.
The abstract does not give per-dataset nDCG@100 values, variance or significance tests for comparison counts, or the behavior of DualEnd when the candidate pool exceeds the context window; these are open questions the body may address. From the abstract alone, a reader cannot judge the spread across backbones or confirm whether the BEIR efficiency gains come with effectiveness changes.