# Selective Prediction

On the corrected CIC-IDS2017 dataset, when labels and confidences from four heterogeneous detectors are corrupted under a threat model, unverified open-weight LLM investigators fall to 0.50–0.72 accuracy with one corrupted detector and to 0.13 or less with three. Conflict-Aware Evidence Verification (CAV) keeps reported confidences out of the vote and, only when tools conflict, tests the leading candidates with a nearest-neighbour trust score computed from raw traffic features. With its prompt frozen before testing, CAV-LLM reaches 0.84–0.88 under three corrupted detectors in both persistent and sporadic scopes, against at most 0.08 for the naive investigator, while calling the LLM on 0.4% of clean traffic.

On the corrected CIC-IDS2017 dataset, when labels and confidences from four heterogeneous detectors are corrupted under a threat model, unverified open-weight LLM investigators fall to 0.50–0.72 accuracy with one corrupted detector and to 0.13 or less with three. Conflict-Aware Evidence Verification (CAV) keeps reported confidences out of the vote and, only when tools conflict, tests the leading candidates with a nearest-neighbour trust score computed from raw traffic features. With its prompt frozen before testing, CAV-LLM reaches 0.84–0.88 under three corrupted detectors in both persistent and sporadic scopes, against at most 0.08 for the naive investigator, while calling the LLM on 0.4% of clean traffic.

High-stakes regression decisions need a point estimate, a risk interval and an option to abstain, yet interval methods and selective prediction each supply only part of that triad. SelPI trains interval, value and two abstention heads jointly under one loss, with a Fréchet penalty tying the two gates' budgets to the joint acceptance rate they deliver. Across 18 tabular benchmarks, interval width predicts large point errors well but is anti-correlated with interval misses, so abstaining on wide intervals discards inputs the interval got right; with the base predictor held fixed, the learned gate beats width thresholding on a proper interval score on all 18 tabular datasets, and the advantage grows with task heteroscedasticity.

High-stakes regression decisions need a point estimate, a risk interval and an option to abstain, yet interval methods and selective prediction each supply only part of that triad. SelPI trains interval, value and two abstention heads jointly under one loss, with a Fréchet penalty tying the two gates' budgets to the joint acceptance rate they deliver. Across 18 tabular benchmarks, interval width predicts large point errors well but is anti-correlated with interval misses, so abstaining on wide intervals discards inputs the interval got right; with the base predictor held fixed, the learned gate beats width thresholding on a proper interval score on all 18 tabular datasets, and the advantage grows with task heteroscedasticity.

Subtle pediatric wrist fractures often appear on radiographs only as a focal cortical interruption or a fine linear lucency, and the epiphyses, growth plates and overlapping bones make these signs hard to separate. MV-CortexGuard compares cortical continuity across frontal and lateral views in a shared anatomical coordinate frame and refers high-risk examinations for expert review. On the internal GRAZPEDWRI-DX test set it reached 52.7% average precision on the size-defined small-fracture subset, 6.6 points above YOLOv10m, and 63.4% AP@50–95 overall.

Subtle pediatric wrist fractures often appear on radiographs only as a focal cortical interruption or a fine linear lucency, and the epiphyses, growth plates and overlapping bones make these signs hard to separate. MV-CortexGuard compares cortical continuity across frontal and lateral views in a shared anatomical coordinate frame and refers high-risk examinations for expert review. On the internal GRAZPEDWRI-DX test set it reached 52.7% average precision on the size-defined small-fracture subset, 6.6 points above YOLOv10m, and 63.4% AP@50–95 overall.

Automated angle-of-progression (AoP) measurement from intrapartum ultrasound is accurate to a few degrees on average, yet a small fraction of frames fails silently by tens of degrees. On the 501 test files of the public IUGC2025 benchmark, an ensemble of six landmark detectors ranked frames by disagreement between detectors; referring the most suspicious 5% for manual review lowered the share of errors above 20 degrees among retained frames from 1.2% to 0.2%, catching five of six. The missed case was one all six detectors underestimated with little spread between them; no difference was detected between angle-space and coordinate-space disagreement at the primary endpoint, and thresholds set on validation frames did not hold their intended referral rate on test frames.

Automated angle-of-progression (AoP) measurement from intrapartum ultrasound is accurate to a few degrees on average, yet a small fraction of frames fails silently by tens of degrees. On the 501 test files of the public IUGC2025 benchmark, an ensemble of six landmark detectors ranked frames by disagreement between detectors; referring the most suspicious 5% for manual review lowered the share of errors above 20 degrees among retained frames from 1.2% to 0.2%, catching five of six. The missed case was one all six detectors underestimated with little spread between them; no difference was detected between angle-space and coordinate-space disagreement at the primary endpoint, and thresholds set on validation frames did not hold their intended referral rate on test frames.

On factual question answering with Qwen3-14B, keeping the generator fixed and training a separate answer selector works better than training the model itself when to abstain. On TriviaQA the selector raises mean coverage from 20.2% under the original calibrated confidence filter to 30.9%, while pooled error among scored answers falls from 23.0% to 14.5%; on NQ-Open its mean utility still stays below always abstaining, so reliable cross-dataset abstention remains unresolved.

On factual question answering with Qwen3-14B, keeping the generator fixed and training a separate answer selector works better than training the model itself when to abstain. On TriviaQA the selector raises mean coverage from 20.2% under the original calibrated confidence filter to 30.9%, while pooled error among scored answers falls from 23.0% to 14.5%; on NQ-Open its mean utility still stays below always abstaining, so reliable cross-dataset abstention remains unresolved.

The work proposes SAUCE, a training-free uncertainty estimator that casts uncertainty in parallel multi-agent reasoning as sequential inference over a latent system-level belief: each round's cross-agent agreement serves as evidence and its generation uncertainty sets the evidence weight, updated through a filtering-style recursion, improving misclassification detection, selective prediction, and calibration across five backbones, five benchmarks, and the Debate and DyLAN protocols.

The work proposes SAUCE, a training-free uncertainty estimator that casts uncertainty in parallel multi-agent reasoning as sequential inference over a latent system-level belief: each round's cross-agent agreement serves as evidence and its generation uncertainty sets the evidence weight, updated through a filtering-style recursion, improving misclassification detection, selective prediction, and calibration across five backbones, five benchmarks, and the Debate and DyLAN protocols.