Robots stop swallowing every experience whole: keeping only human-controlled segments of successful interventions lifts precision insertion success to 88.24%

Continued robot policy improvement no longer treats all experience collected during deployment as equally useful; it first judges which segments are worth learning from, then decides which data enter training. RECAP trained the policy on accumulated experience with a binary advantage label as an additional input; Evo-RL instead uses estimated advantages to select experience for policy updates and increases the sampling weight of selected experience. On the PiperX screw-insertion task, first-round Evo-RL reached 88.24% attempt success rate versus RECAP's 57.78%, with throughput of 60.61 versus 35.54 successful stages per hour.

The method places human corrections and advantage-based selection in one iterative loop: corrections supply behavior for states where the policy gets stuck during deployment, while advantage selection steers training toward promising behavior in the available data. Human interventions alone (SFT + Int.) raised attempt success rate from SFT's 22.73% to 78.72%, advantage selection alone (SFT + RL) reached 36.99%, and combining them reached 88.24%. All variants start from the same policy trained on demonstrations for 2.5 epochs and are evaluated on PiperX under the same protocol; a second Evo-RL round raised attempt success rate further to 92.16% and throughput to 67.14.

Training retains only the human-controlled portions of successful intervention trajectories, excluding policy-controlled portions to avoid learning from actions likely to cause failures or prolong task completion. Intervention samples usually form only a small fraction of collected data, and IWR addresses this imbalance by increasing their weight; Evo-RL goes further by splitting trajectories by control source and sampling action chunks only within contiguous human-controlled segments. On the SO-101 folding task, full Evo-RL raised attempt success rate from SFT's 55.56% to 76.67% and throughput from 75.78 to 90.00 successful stages per hour; at the spreading stage, successful completions rose from 17 to 19 while attempts fell from 76 to 36.

The report covers two real manipulation tasks, PiperX screw insertion and SO-101 folding, with policies starting from a demonstration-fine-tuned checkpoint and deployed with real-time action chunking. It is intended for researchers and engineers interested in real-world robot reinforcement learning and the use of human-intervention data. The next step explicitly stated is repeating the data collection and training cycle, and a second Evo-RL round raised PiperX attempt success rate from 88.24% to 92.16%.

The value models are task-specific and vision-only, so each task requires its own model. The material reports policy fine-tuning for first-round Evo-RL at about 4 h 52 min and 1,511 steps, and value training at about 2 h 19 min, all on multi-GPU hardware; these times correspond to a specific hardware configuration. A second iteration improved PiperX further, but the material does not state whether additional rounds continue to help.