Agents don't need a new model to get better: rewriting the runtime harness lifts GLM-5 by 11.5 points on office tasks
Synopsis
An agent's performance depends not only on its underlying model but on its runtime harness: prompts, memory, skills, tools and orchestration. Harness2 makes harness improvement a two-level recursion. At the task level, a contrastive propose-probe-compose step compares execution behavior before and after edits without ground-truth answers; at the harnessing level, that process is distilled into an experience library that guides later tasks. On WorkBuddy-Bench, JobBench and Harvey's Legal Agent Benchmark, a single step improved both GLM-5 and Gemini 3.5 Flash across four model-harness configurations, by up to 11.5 points, 4.5 points and 11.8 percentage points.
Interpretation
An agent's runtime harness can be rewritten automatically over a stream of deployment tasks, rather than fixed in advance on training tasks. Earlier approaches either selected a harness offline against fixed training tasks, unit tests or validation feedback, or revised it only for the current task, so the value of an edit rarely carried over when the task changed. Across three professional benchmarks and four model-harness configurations, a single improvement step beat each base harness for both GLM-5 and Gemini 3.5 Flash, with gains of up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench and 11.8 percentage points on the legal benchmark.
The improvement process itself becomes a learning target: after each task, the adopted harness and the lessons from refining it enter an experience library for later tasks. Prior methods typically carried a successful execution harness into the next task batch, or improved only the current task's harness, without updating the layer that decides how to improve harnesses. Ablations show that removing the library lowers WorkBuddy-Bench and JobBench by 3.1 and 1.8 points; removing scoped candidate editing costs 2.9 and 3.7 points, and removing composition costs 1.6 and 1.9 points.
Task-level recursion can scale exploration in parallel or in sequence, adding gains beyond a single step. A single step already exceeded the base harness, but whether a larger exploration budget keeps helping had to be separated from simply spending more test-time compute. Parallel task-level recursion added 2.9 to 3.9 points on WorkBuddy-Bench, 3.0 to 4.2 points on JobBench and 2.6 to 6.7 percentage points in LAB Partial pass rate over a single step.
Perspective
The report covers evaluation on three professional benchmarks: office-style deliverables in WorkBuddy-Bench, end-to-end professional tasks in JobBench, and Harvey's Legal Agent Benchmark, reported as Partial, the per-criterion pass rate. Executors are frozen Gemini 3.5 Flash and GLM-5 under the off-the-shelf OpenCode and Codex harnesses, with Claude Opus 4.8 as the default harness editor. The improvement library starts empty for each benchmark stream and is updated after each task; completed tasks are not replayed.
Recursive refinement requires extra executions and editor sessions, so a larger recursion budget adds inference cost; parallelism can reduce latency but does not remove that computation. Without ground-truth supervision, refinement relies on behavioral contrasts and model judgment, which are sensitive to execution variability and shared model biases and do not guarantee correctness or that an adopted change transfers to another task. The failure analysis also lists selection preferring a weaker candidate, incorrect premises propagating across candidates, and outcome variability plus missing judge verdicts complicating attribution.