ActiveSaddler co-evolves training scenarios with the harness, raising Pass@1 by 4.4 and 7.5 points on GAIA2 and Terminal-Bench 2.0
The work formulates training-scenario selection in offline harness optimization as an automated curriculum-learning problem and introduces ActiveSaddler, which models the curriculum as a non-stationary bandit with failure-pattern arms, estimates each arm's potential learning progress, and adaptively balances revisiting known weaknesses against exploring unseen scenarios, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer with a scenario order fixed before optimization on GAIA2 and Terminal-Bench 2.0 respectively.