Teaching a model to say "I don't know": a separate answer selector beats retraining the policy, lifting TriviaQA coverage from 20.2% to 30.9%
Synopsis
On factual question answering with Qwen3-14B, keeping the generator fixed and training a separate answer selector works better than training the model itself when to abstain. On TriviaQA the selector raises mean coverage from 20.2% under the original calibrated confidence filter to 30.9%, while pooled error among scored answers falls from 23.0% to 14.5%; on NQ-Open its mean utility still stays below always abstaining, so reliable cross-dataset abstention remains unresolved.
Interpretation
In short factual QA, letting a separate component decide whether an answer should be released improves useful abstention more than changing the model that generates answers. The prior route gave the original model a confidence threshold and released answers by calibrated confidence; that filter released only 20.2% of TriviaQA questions. On 2,000 TriviaQA test questions, the selector raised mean coverage to 30.9% and cut the pooled error rate among scored answers from 23.0% to 14.5%, with mean utility rising from 0.02418 to 0.04248; the paired gain of 0.01830 has a 99% bootstrap interval of [0.01041, 0.02593] and is positive for each of three seeds.
The selector does not retrain the generator; it learns a second correctness score over the generator's candidate answers. The original model scores candidates with P(True) and applies logistic calibration, which only remaps one scalar score; the selector uses a LoRA adapter to learn exact-match correctness from the question and candidate alone. Training used 2,000 TriviaQA questions with one greedy and three sampled candidates each, 8,000 examples and 500 updates; on 1,784 valid TriviaQA candidates, ROC AUC rose from 0.779 for the original score to a mean of 0.852 across selector seeds, and calibrated Brier fell from 0.189 to 0.154.
Giving the same 8,000 training labels to a logistic model fitted on the original score does not recover the selector's gain. That control applies an affine transform to a fixed scalar score and cannot learn new question-candidate features. After calibration, the selector's mean utility advantage over that control is 0.01834, with a 99% interval of [0.01041, 0.02602].
The tested reinforcement-learning changes show no consistent gain, and transfer to another dataset does not hold up. All three RL contrasts have positive mean utility differences, but adjusted intervals include zero and directions vary by seed; on NQ the selector beats the original calibrated filter yet stays below the zero utility of always abstaining. On NQ the selector's difference from the original calibrated filter is 0.01878 with a descriptive 95% interval of [0.00605, 0.03097]; at threshold 0.85 the original filter releases one answer, which is incorrect, and the selector's utility is lower there.
Perspective
The results cover Qwen3-14B on short English factual questions without retrieval, graded by exact match, with a 2,000-question TriviaQA training pool, 2,000 TriviaQA and 500 NQ-Open test questions, and primary thresholds 0.65 and 0.85 held out from policy training. The selector and its calibration are learned on TriviaQA and transferred unchanged to NQ. Readers interested in other model sizes, languages, or retrieval settings would need separate validation.
The reinforcement-learning contrasts have intervals that include zero and directions that vary by seed, so whether those changes give a consistent gain is not established. On NQ the selector's mean utility remains below always abstaining, and its relative advantage reverses at threshold 0.85, where the original filter's result rests on a single incorrect answer. The human review covers only 150 response units with AI-proposed labels and one reviewer, not two independent annotators, so it cannot estimate grading error over the full test sets.