Agent benchmarks get harder without new tasks: recursive environment rewriting cuts six models' success across the board

An agent benchmark's difficulty can come from the environment itself rather than from replacing tasks. RES treats an existing benchmark environment as a scalable dimension, rewriting how information is accessed, how resources relate, and when actions become available, while keeping the core objective, underlying facts and completion requirements fixed. Earlier work on expanding agent experience mostly synthesized new tasks or built environments from scratch; RES compares against the original environment of the same task, forming a paired comparison rather than a comparison across different task sets. Across 294 initial tasks from Claw-Eval-Live, Workspace-Bench-Lite and Terminal-Bench 2.1, task success fell in all 18 model-benchmark comparisons for six evaluation models; on Claw-Eval-Live, DeepSeek-V4.1-Flash dropped from 82.86% to 5.71% and Claude Opus 5 from 86.67% to 28.57%.

Accepted changes accumulate along a task lineage, so later transformations can reuse existing access paths and attach new prerequisites to inherited operations, lengthening the execution path around the same objective. Ablations separate recursive inheritance from independent enhancement: both receive the same nominal construction opportunities, but each independent attempt starts from the original environment while recursive construction continues from the previously accepted environment. On 15 tasks, recursive construction reached 6.7% pass@1 versus 93.3% for independent enhancement, with native scores of 14.40 versus 88.73 and roughly 2.5 times the mean target-token expenditure; removing cross-operator references and preparation prerequisites raised pass@1 from 0.0% to 100.0% and cut mean turns from 50.0 to 19.1.

Trajectories collected in scaled environments can be used for training and transfer back to the original tasks. Both training conditions start from the same Base checkpoint and use equal numbers of trajectories with identical optimization settings; they differ only in whether trajectories come from original ClawGym environments or RES-scaled environments. For all three Qwen3.5 Base backbones, RES-trajectory SFT outperformed original-trajectory SFT in all nine model-benchmark comparisons, with gains of 7.00 to 14.61 percentage points; averaged over the three backbones, gains were 10.16, 10.00 and 10.86 percentage points on Claw-Eval-Live, Workspace-Bench-Lite and Terminal-Bench 2.1.

The report covers 294 initial tasks from three benchmarks, evaluated after environments are frozen with one attempt per task, a 50-turn limit and high reasoning effort, holding tool permissions and scoring fixed within each pair. Transformations are proposed by GPT-5.6 Sol, the scaling-stage target is DeepSeek-V4.1-Flash, and recursion accepts at most ten transformations. The training comparison uses three Qwen3.5 Base backbones with 8,428 trajectories each and evaluates on the original tasks. Manual review covers 20 matched original/scaled pairs per benchmark, 60 task lineages and 120 endpoint environments in total.

Construction cost counts only executions associated with accepted children, excluding branches that produced no child, target solving and grading, so execution counts alone do not capture scaling cost. In the depth analysis, higher-budget summaries combine the final environments of many stopped lineages with the few that continue, and depth and endpoint evaluations use separate batches. The training comparison uses no held-out validation split and evaluates only the final checkpoint. Manual review found no objective drift, answer leakage or scoring bypass in the reviewed samples, but that review is limited to a sample.