# Confidence Calibration

Agentic reinforcement learning shifts the reasoning trajectories of large language models, and trajectory-based confidence estimators trained on a base model lose much of their ranking ability after that shift. StaR-TTA treats trajectory stability as a label-free reliability signal: it learns the link between stability and correctness on labeled base trajectories, then uses stability rankings from unlabeled RL trajectories to correct confidence at test time. Across four QA benchmarks it reaches an average AUROC of 0.7208, with gains on unseen datasets, different model families and scales, and both PPO and GRPO.

Agentic reinforcement learning shifts the reasoning trajectories of large language models, and trajectory-based confidence estimators trained on a base model lose much of their ranking ability after that shift. StaR-TTA treats trajectory stability as a label-free reliability signal: it learns the link between stability and correctness on labeled base trajectories, then uses stability rankings from unlabeled RL trajectories to correct confidence at test time. Across four QA benchmarks it reaches an average AUROC of 0.7208, with gains on unseen datasets, different model families and scales, and both PPO and GRPO.

The work proposes RL-ARC, a calibration-aware reinforcement learning framework that, on top of RLVR, elicits both the model's confidence in its reasoning process and its confidence in its answer, using reasoning confidence as an auxiliary signal: reasoning-guided regularization when the prediction is correct and an overconfidence penalty when it is incorrect; across mathematical reasoning and out-of-distribution complex reasoning and factual QA benchmarks, RL-ARC improves calibration metrics over existing calibration-aware training methods while keeping accuracy comparable to RLVR, which optimizes correctness alone.

The work proposes RL-ARC, a calibration-aware reinforcement learning framework that, on top of RLVR, elicits both the model's confidence in its reasoning process and its confidence in its answer, using reasoning confidence as an auxiliary signal: reasoning-guided regularization when the prediction is correct and an overconfidence penalty when it is incorrect; across mathematical reasoning and out-of-distribution complex reasoning and factual QA benchmarks, RL-ARC improves calibration metrics over existing calibration-aware training methods while keeping accuracy comparable to RLVR, which optimizes correctness alone.

The work introduces ConRad, a reinforcement learning framework that fine-tunes medical large vision-language models with the GRPO algorithm and a logarithmic scoring-rule reward so they emit verbalized confidence alongside radiology reports at report or sentence level, reducing report-level ECE to 0.106 and sentence-level ECE to 0.239 on MIMIC-CXR and improving zero-shot IU-Xray ECE from 0.310 to 0.034 while keeping report quality (GREEN) stable.

The work introduces ConRad, a reinforcement learning framework that fine-tunes medical large vision-language models with the GRPO algorithm and a logarithmic scoring-rule reward so they emit verbalized confidence alongside radiology reports at report or sentence level, reducing report-level ECE to 0.106 and sentence-level ECE to 0.239 on MIMIC-CXR and improving zero-shot IU-Xray ECE from 0.310 to 0.034 while keeping report quality (GREEN) stable.