Passing tests is not enough: structural scoring picks 1,024 agent trajectories and lifts SWE-Bench Verified pass@1 to 27.6%

Coding-agent supervised fine-tuning data can be selected by the structure of the interaction process rather than only by whether the final patch passes the benchmark. The common practice keeps trajectories whose final patches pass the benchmark test suite, yet such trajectories may still repeatedly inspect the same files, interrupt stalled shell commands, or postpone meaningful edits until late in the interaction budget. On 155,144 successful trajectories from CoderForge-Preview, the same Qwen3-Coder-30B-A3B-Instruct model and identical LoRA recipe were trained on 1,024 top-ranked, bottom-ranked, and random trajectories; the top subset reached 27.6% pass@1 on SWE-Bench Verified, versus 24.8% for random and 21.8% for bottom-ranked.

The score reads only structure already present in the event log, with no additional language-model judge. Perplexity-style scores are confounded by length and tokenization, and LLM-based judges are costly at the scale of hundreds of thousands of trajectories and billions of tokens while not characterizing how an agent organizes its search over time. Seven components cover observation entropy collapse, progressive scope narrowing, intent-action consistency, tool transition entropy, solution commitment point, post-error diversity, and a dirty-pattern guardrail, combined with fixed weights; on an AMD EPYC 7413 a single worker processed 3,462 trajectories in 149.2 seconds, about 43 ms per trajectory.

The gain from structural selection is larger on harder tasks. On easy tasks the top subset's advantage over the random subset is only 1.6 points. Broken down by difficulty, the top-versus-random gap is 3.4 points on medium tasks and 4.5 points on hard tasks, where both reach 6.7% pass@1.

The scorer targets OpenHands-style coding-agent trajectories, taking as input a sequence of steps with action type, thought text, tool observation, touched file paths, and an error flag, and returning a scalar used to rank successful trajectories. It fits fixed-budget supervised fine-tuning data selection and corpus-scale repeated scoring; on a 3,462-trajectory benchmark a single worker runs at about 43 ms per trajectory, and four workers cover all 258,134 unique trajectories in about 49 minutes.

The experiments use CoderForge-Preview trajectories generated by a single teacher model under the OpenHands scaffold, and the component parsers and thresholds may require recalibration for different action spaces, multilingual traces, or other step granularities, as may the hand-set component weights. Training trajectories are capped at 32,768 tokens, excluding a long-trajectory tail that may contain useful recovery behavior, and each condition is trained once, so the reported pass@1 differences carry no multi-seed variance estimates. Downstream evaluation is limited to the 30B model; preliminary 7B experiments did not show stable selector differences, and larger backbones remain untested.