Agent RL stops relying on its own luck: failed rollouts get corrected mid-sampling, lifting navigation pass rate from 35% to 57%

Agent reinforcement learning can learn from successful trajectories that the current policy almost never samples, pushing past its original capability boundary. Existing agent RL is limited by its on-policy nature: it mainly amplifies behaviors the policy already supports, and when tasks lie beyond the capability boundary successful trajectories become extremely sparse, so training is unstable or collapses. Across five benchmarks RTI consistently beats GRPO, GSPO, AEPO and the teacher-guided OPD and LUFFY; with the Qwen3-8B backbone it improves Pass@1 over GRPO by 24.56 points on -Telecom and 22.00 points on Navigation.

Intervention fires only when every trajectory in a group fails, then a validator, corrector and verifier turn the failed rollout into a successful one, a perplexity gate drops corrections that drift too far, and token reweighting concentrates updates on the tokens that changed the reasoning direction. Corrected trajectories come from an intervention-induced behavior policy that differs from the original policy; using them directly pushes importance ratios outside the clipping range and suppresses useful learning signals, while low-probability actions give high-variance gradients. Ablations on Navigation and -Telecom with Qwen3-8B show that removing the integrity verifier drops -Telecom back to roughly GRPO level and Navigation Pass@1 from 57.33 to 26.00, removing the perplexity gate drops -Telecom Pass@1 from 61.40 to 54.39, and removing token reweighting lowers all sixteen metrics slightly.

The report is for researchers and engineers working on agent reinforcement learning and tool-use training. RTI is validated on five benchmarks: a self-built multi-waypoint Navigation task, the Telecom, Retail and Airline domains of -Bench, and Claw-Eval-Hard20; backbones are Qwen3-8B and Qwen3.5-9B, and the three intervention roles default to the policy model itself. Training dynamics show intervention supplies successful trajectories early and the policy increasingly succeeds without intervention as learning proceeds.

Intervention trajectories remain inherently off-policy and guide policy optimization effectively only after trajectories with excessive distributional shifts are filtered out; a threshold that is too small discards too many corrective trajectories, while one that is too large admits severely shifted interventions and destabilizes optimization. Using the much larger Qwen3.5-397B-A17B as the intervening model makes corrections deviate further from the policy distribution and harder to learn from, so standard self-intervention performs best. Gains on Rest-OOD are comparatively modest, partly because that set is reported without difficulty screening and a non-trivial fraction of its tasks are effectively infeasible or not reliably evaluable under the evaluation environment.