RSI-Env lets an agent rewrite both model and harness, then hands the next round to the accepted version, with gains even without benchmark feedback
Synopsis
RSI-Env places model training and harness modification in one persistent sandbox, with an independent Host Controller handling services, validity checks, versioning and capability evaluation, and in Self mode each accepted model–harness version takes over the next improvement round. Starting from Qwen3.8-27B with a minimal harness, Open-General Self, which receives no benchmark feedback, raises GPQA Diamond accuracy from 65.2% to 73.2% and AIME 2026 from 53.3% to 56.7%; Knowledge Self lifts SimpleQA from 82.0% to 85.0% and then 88.0%, while the same runs fall back on other benchmarks.
Interpretation
An agent can choose its own improvement direction with no named benchmark and no benchmark feedback, treating model weights and harness code as editable together. Earlier self-improvement work typically fixed the optimization objective, editable components, update procedure or evaluation feedback in advance, which made it hard to see how an agent formulates its own improvement agenda. Starting from Qwen3.8-27B with a minimal harness, Open-General Self made one harness update with model weights unchanged (H1/M0), raising GPQA Diamond accuracy from 65.2% to 73.2% and AIME 2026 from 53.3% to 56.7%.
An accepted model–harness version takes over the next research round, so improvement carries across versions instead of restarting from the same initial system. Environments that already offered training and evaluation services did not, in their reported experiments, test updated systems taking over subsequent research rounds. Knowledge Self went from H0/M0 at 82.0% SimpleQA to H1/M0 at 85.0% after a harness update, then H1/M0 built 48 prompt–completion pairs to train M1, and H1/M1 reached 88.0%.
Improvement work is split between an editable sandbox where the agent experiments freely and a Host Controller it cannot edit, which manages services, checks, versioning and independent evaluation. Evaluation criteria and version records sit outside the editable environment, and acceptance of an intermediate update depends only on validity and executability. Model training goes through the Tinker SDK train-sft interface, which accepts JSONL prompt–completion pairs and returns a registered model candidate, while capability changes are measured controller-side and are not the acceptance criterion for intermediate updates.
Perspective
The demonstrated scope is a common starting point of Qwen3.8-27B with a minimal harness, comparing Self with two fixed external Teachers on seven tasks under the Open-General, Capability-Directed and Benchmark-Guided settings, with each run allotted ten rounds of three hours, thirty hours in total. It suits researchers working on self-improvement infrastructure and evaluation design.
Gains are uneven: Open-General Self falls to 77.2% on SimpleQA, and Knowledge Self falls to 53.0% on GPQA Diamond. Benchmark-Guided beats the corresponding Capability-Directed settings on AIME 2026 and Terminal-Bench 2.1 but scores lower on SimpleQA, and external Teachers do not consistently outperform Self, nor are stronger models consistently better improvers.