Skip to main content

# Terminal-Bench 2

2 related articles

Subscribe to updates

arXiv Oct 5, 2026
SWR self-supervises training environments from 500 scientific software workflows, lifting Qwen3.8-27B on Terminal-Bench 2 from 47.94% to 53.56%
The authors introduce software-in-the-loop reconstruction (SWR), a self-supervised framework that draws reference outputs and verification targets from existing software workflows, instantiated with 500 workflows across 46 software families and six domains, where Qwen3.8-Max solves 838 tasks over three attempts and yields 1,422 verified trajectories oversampled into 3,000 reconstruction-only training examples, and supervised fine-tuning raises Qwen3.8-27B mean Terminal-Bench 2 performance from 47.94% to 53.56% while achieving the highest mean among four matched-token corpus controls on all four reported evaluations.
The authors introduce software-in-the-loop reconstruction (SWR), a self-supervised framework that draws reference outputs and verification targets from existing software workflows, instantiated with 500 workflows across 46 software families and six domains, where Qwen3.8-Max solves 838 tasks over three attempts and yields 1,422 verified trajectories oversampled into 3,000 reconstruction-only training examples, and supervised fine-tuning raises Qwen3.8-27B mean Terminal-Bench 2 performance from 47.94% to 53.56% while achieving the highest mean among four matched-token corpus controls on all four reported evaluations.
Hong Kong Polytechnic University University of Georgia Indiana University National University of Singapore University of Illinois Chicago University of Maryland, College Park
Zhongzhi Li（First author）
Software-in-the-loop reconstruction Terminal agents Terminal-Bench 2 Supervised Fine-tuning
arXiv Oct 2, 2026
ActiveSaddler co-evolves training scenarios with the harness, raising Pass@1 by 4.4 and 7.5 points on GAIA2 and Terminal-Bench 2.0
The work formulates training-scenario selection in offline harness optimization as an automated curriculum-learning problem and introduces ActiveSaddler, which models the curriculum as a non-stationary bandit with failure-pattern arms, estimates each arm's potential learning progress, and adaptively balances revisiting known weaknesses against exploring unseen scenarios, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer with a scenario order fixed before optimization on GAIA2 and Terminal-Bench 2.0 respectively.
The work formulates training-scenario selection in offline harness optimization as an automated curriculum-learning problem and introduces ActiveSaddler, which models the curriculum as a non-stationary bandit with failure-pattern arms, estimates each arm's potential learning progress, and adaptively balances revisiting known weaknesses against exploring unseen scenarios, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer with a scenario order fixed before optimization on GAIA2 and Terminal-Bench 2.0 respectively.
Microsoft
ActiveSaddler Curriculum Learning Agent Harness Non-Stationary Bandits GAIA2

You’re all caught up