Rejected patches stop being wasted: failure feedback enters training, lifting SWE-bench first-attempt resolution from 38.9% to 41.7%
Synopsis
Repository-level software engineering agents previously treated each attempt's pass or fail as a terminal reward, discarding the failed patch and verifier feedback. FC-SWE restores the repository to its original task state after a patch fails verification, uses the failed patch and verifier diagnostics as context for a recovery trajectory, keeps each trajectory's own verifier outcome, and compares only the initial and recovery trajectories actually executed for the same issue. On all 500 SWE-bench Verified tasks with the same Qwen3.5-4B policy, first-attempt resolution rises from GRPO's 38.9% to 41.7% and two-attempt cumulative resolution from 48.5% to 52.8%; trained with at most two attempts per chain, the same policy reaches 70.7% with eleven test-time attempts.
Interpretation
Failed patches and verifier feedback now enter the context of a later attempt instead of leaving only a failure label. Earlier training sampled each attempt independently, so a failed patch neither triggered nor conditioned another trajectory in the training group. On all 500 SWE-bench Verified tasks with the same Qwen3.5-4B policy, FC-SWE reaches 41.7% Resolved@1 and 52.8% Resolved@2, against 38.9% and 48.5% for GRPO and 27.9% and 36.3% for the initial policy.
Each trajectory receives only its own verifier outcome, so a later recovery success does not relabel the earlier rejected patch as correct. Broadcasting the final chain outcome to every executed trajectory would assign a positive reward to the rejected initial patch solely because a later invocation succeeded. Removing trajectory-local rewards in favor of the shared final outcome lowers conditional recovery from 20.4% to 17.7% and Resolved@2 from 54.8% to 51.8%.
Advantage estimation forms its comparison group from the initial and recovery trajectories actually executed for the same issue, excluding unexecuted recovery slots. Grouping by attempt index would omit cross-attempt comparisons, while counting unexecuted slots would contaminate the group statistics. Trained with at most two attempts per chain, the same policy reaches 70.7% cumulative resolution with up to eleven verifier-guided test-time attempts, against 67.3% for GRPO and 61.1% for the initial policy.
Perspective
Evaluation covers all 500 SWE-bench Verified tasks with SWE-agent and the official SWE-bench verification procedure; the controller observes benchmark-test outcomes and diagnostic feedback between attempts and stops after success, so multi-attempt numbers measure verifier-assisted recovery rather than independent-sampling Pass@. The controlled training comparison uses Qwen3.5-4B, with training data from ten R2E-Gym repositories that do not overlap SWE-bench Verified. Using verifiers available during deployment, and separating the verifier that guides recovery from held-out tests used for final scoring, is the next step the source proposes.
Evaluation assumes benchmark-test feedback is available after each attempt, and passing a finite test suite does not guarantee semantic correctness or complete satisfaction of an issue description. The controlled training study uses one model and one configuration; multiple decoding seeds characterize inference variation rather than variation across independently trained policies, some ablations use a single decoding seed, and the penalty-removal comparison is confined to shared-reward training. FC-SWE generates additional trajectories during training, so the matched-task, matched-initial-group comparison is not matched in total training compute; inference token costs are estimates, not measured end-to-end latency.
