Tail calibration flags when a larger Best-of-N budget backfires, and code generation is where it shows
Synopsis
Best-of-N inference returns the candidate with the highest proxy reward, so a larger budget reads the extreme tail of the reward distribution rather than its average ranking quality. The work writes the BoN value as a Beta-weighted integral of conditional true reward over within-prompt score quantiles, showing that BoN depends on scores only through their ranks and can decline only when the calibration curve in the tail turns downward. On public candidate pools, negative tail excess concentrates in code generation, and a cross-fitted tail statistic predicts whether a budget cap pays off on held-out data about as well as rank accuracy.
Interpretation
The quality of the candidate Best-of-N returns can rise and then fall as the sampling budget grows, and that decline is set only by the calibration shape of the top score region, not by the score distribution as a whole. Reward models are usually graded by preference accuracy, calibration or mean margin, all averaged over the region selection never visits, so a team can pass every leaderboard check and still push a budget past its own optimum. Across 162 synthetic cells, Gaussian additive errors give a monotone BoN curve, Laplace errors reach a plateau, and heavy-tailed errors show a statistically supported downturn at all six correlations, matching the prediction by error tail class.
Mapping scores to within-prompt quantiles makes the BoN value a Beta-weighted integral of conditional true reward over that quantile, so the reward model is used only through its ranks. This representation turns the budget question into a question about one curve: whether BoN can decline depends on whether the calibration curve bends down, and where it declines depends on how close to the top that bend sits. Among the 65 learned reward-model cells from the public PPE pools, all six negative top-12.5% tail excesses occur on MBPP+, and the other four tasks have none.
A tail excess statistic confined to the top score region, paired with a cross-fitted reward response curve, yields a budget cap for a named task and model. Common mitigations such as KL penalties, soft selection or ensembling apply a uniform rate whether or not a task is at risk, whereas this audit caps only the task and model where a tail downturn is detected. Of 130 cross-fitted folds, nine calibration-selected caps have a positive held-out interval, and tail risk predicts that event with ROC AUC 0.983, interval [0.954, 0.998], while the paired difference from rank accuracy includes zero.
Perspective
The demonstrated scope is BoN selection on labeled candidate pools: synthetic experiments use a bounded uniform prior with Gaussian, Laplace, Student-t and multiplicative errors, the controlled study uses fixed candidates generated by Qwen2.5-7B-Instruct on MATH-500 and GSM8K, and external validation uses 65 learned reward-model cells from four base policies on five PPE tasks. A reader should treat it as a pre-deployment audit tool, not a replacement for ranking metrics.
Negative tail excess is rare in the public pools and the average-precision intervals are wide, and the authors explicitly do not claim that bulk ranking quality cannot predict the problem in practice. In the controlled MATH study the answer-only failure cannot be attributed to missing question semantics alone, since the chat role and template change is an important confound, and GSM8K shows no supported downturn in either original format. Sensitivity checks on tail width and tie order preserve the main pattern, but missing endpoint normalization and policy likelihoods prevent same-pool comparisons with Best-of-Tails and ITP.