Agent benchmarks get harder without new tasks: recursive environment rewriting cuts six models' success across the board
The difficulty of agent evaluation now comes from the environment rather than from swapping tasks. Recursive Environment Scaling (RES) rewrites information access, resource relations and execution dependencies while preserving each task's objective and completion requirements, validating every candidate by execution and letting accepted changes be inherited across rounds. On 294 initial tasks from three benchmarks, task success fell for all six evaluation models, by 57.14 to 77.14 percentage points on Claw-Eval-Live; with equal trajectory counts and identical optimization settings, fine-tuning on RES trajectories beat original-trajectory SFT in all nine model-benchmark comparisons by 7.00 to 14.61 percentage points.