Coding agent teams don't need strong models everywhere: cheap planners nearly match an all-strong team

Organizing coding agents into a team passes more behavioral tests than a single agent on from-scratch program rebuilding, but runs at clearly higher cost. A single agent scored lower on the same tasks, while team setups cost four to seven times as much as the corresponding solo agent. On 200 ProgramBench programs with the same minimal prompt, a blocking Sol team passed 61.8% of tests on average against 52.1% for a solo Sol agent; with the PIV prompt the solo agent rose to 57.7% while the blocking team stayed about 5 points ahead at 63.1%.

The stage worth extra agents is exploration and planning, not implementation or verification. Delegating only planning matched delegating all three stages while halving the cost. Among asynchronous teams, planning-only teams passed 63.6% on average versus 63.2% for teams delegating all three stages, at $249 versus $510; measured as reward gained per $100 of added cost, planning was about twice as efficient as the next-best stage.

Planning workers can run on a cheaper model, but the lead still needs a strong one. Swapping planning workers from Sol to the cheaper Luna cost only a few points, while upgrading the lead from Luna to Terra to Sol raised reward at every step. A Sol lead with Luna planners passed 60.5% against 63.6% for the all-Sol team; teams led by Luna, Terra, and Sol reached 42.5%, 51.6%, and 60.5%, each beating a solo agent on its lead's model, but a Terra lead with Luna workers still fell short of a solo Sol agent (51.6% versus 57.7%).

Adding planning workers keeps adding information, while adding implementation workers does not. Planning teams improved as they grew from two to sixteen Luna workers, whereas implementation and verification teams slipped slightly after four. Planning teams rose from 59.6% with two workers to 61.8% with sixteen, and sixteen Luna planners cost $155; each planner ran the reference program about seven times regardless of team size, so team-wide runs grew from 15 to 109 and reports from 17,000 to 75,000 characters, while implementer assignments shrank from about 1,400 characters per worker to 440.

The conclusions rest on ProgramBench, a benchmark of rebuilding programs from scratch in which the agent must probe the original executable to discover its interface and edge cases, so the planning gain is tied to that setting. The authors also ran Commit0-Lite, 16 Python libraries whose repository skeleton, documentation, and unit tests already state what to build. Readers interested in settings where the specification is given should treat the Commit0 results separately from the ProgramBench results.

Commit0-Lite has only 16 tasks and frontier models are close to saturating it, so the authors treat it as suggestive rather than conclusive. The cross-model comparison ran one trial per configuration on a 30-task subset, so differences of a few points may be noise. In the communication experiment Luna teams lost 4 to 7 points, and the authors offer two possible explanations: interim updates overwhelm a weaker lead, or about half of Luna workers' send_message calls were malformed and rejected by the harness. They also note that a team helps only if the lead chooses to delegate and makes good use of what comes back, and whether that ability emerges with scale remains open.