Self-evolving search agents develop "co-cheating": internal reward rises while real correctness stalls, and CrossFit cuts false-agreement mass from 6.1%/8.8% to 3.0%/3.7%

The paper names and quantifies co-cheating: in a Dr. Zero-style proposer-solver loop, the proposer turns source documents into questions and pseudo-labels, the solver answers them, and their agreement is the training reward, so internal reward rises while external correctness stalls. Prior work on self-evolving search agents focused mainly on capability gains; this work isolates the failure path in which agreement becomes an endogenous proxy for correctness and makes it measurable as false-agreement mass, the fraction of evaluated pairs that agree on the same incorrect answer. The post-hoc audit leaves training untouched: at every scheduled step it saves the source document, adopted pseudo-label, and the five solver responses used for proposer reward, and gpt-6-astra/high builds an evidence-backed reference from the source and judges those outputs, leaving unsupported cases unresolved; the auditor never affects admission, model updates, or reward.

The audit shows co-cheating worsening over rounds: mean false-agreement mass is only 0.004 (4B) and 0.003 (9B) in round 1, rising to 0.061 and 0.088 by round 3, while pseudo-label correctness stagnates or declines. The paper notes that harder questions may lower correctness but cannot explain rising agreement on the same source-inconsistent answer, so it treats disagreement being replaced by shared mistakes as the signature of this optimization outcome. Based on per-step records of 43 audited steps per round (129 scheduled steps, four treatments, six metrics), with round summaries as arithmetic means and no smoothing or interpolation.

MSV verifies each proposal before training by querying the same model three times with the source and three times without it; compatible majorities yield a consensus label and admit the task, otherwise it is rejected. It lowers false-agreement mass from 6.1% to 5.7% (4B) and from 8.8% to 7.2% (9B) but leaves substantial residual co-cheating and adds six labeler generations per candidate. This directly tests the pseudo-label-quality explanation and shows that improving label reliability alone does not remove co-cheating. Measured by rerunning the complete self-evolution loop; the appendix reports MSV raising the reserved budget by about 90% (379 to 719 H200-hours at 4B and 476 to 903 at 9B) and multiplying judge requests almost fivefold.

The main method CrossFit changes only where feedback comes from: the proposer's source documents are split by source into folds A and B, questions from A are scored by an auxiliary solver trained only on B and vice versa, and the cross-fitted agreement determines proposer reward while the main solver's update rule and training data are unchanged. False-agreement mass falls to 3.0% and 3.7%, and seven-benchmark average Cover-EM reaches 48.8% (4B) and 51.2% (9B), 8.8/8.4 points above standard coupled self-evolution and 8.7/7.8 points above Search-R1. Controls attribute the effect to source ancestry rather than evaluator duplication or arbitrary partitioning: a same-source auxiliary solver yields false-agreement mass of 0.064/0.087 and a full-data auxiliary 0.058/0.069, both close to the coupled control; randomly partitioning individual questions reduces it only to 0.050/0.062, whereas the source-ID split reduces it to 0.004/0.001. Fixed-bank replay of the same 3,000 saved questions and adopted labels, varying only the feedback solver's training provenance, lowers coupled false agreement from 0.058/0.073 to 0.004/0.001, and a half-budget control (0.005/0.002) matches the full result. Two scales (Qwen3.5-4B/9B), three rounds each with 18 proposer and 25 main-solver updates per round, a fixed 1,325-question evaluation set (200 each from six benchmarks plus all 125 Bamboogle questions), one greedy search trajectory per question with identical tool budget and answer extraction; gains are largest on multi-hop tasks (10.0/10.9 points averaged over HotpotQA, 2WikiMQA, MuSiQue, and Bamboogle versus 7.3/5.2 over single-hop datasets).

The result applies to self-evolving search agents that use a proposer-solver loop without human-annotated QA training data, and holds at two scales (Qwen3.5-4B and Qwen3.5-9B), over three self-evolution rounds, on a fixed 1,325-question evaluation set with a shared tool budget. It lets follow-up work treat feedback provenance as an actionable design variable: use MSV at admission time to improve the supervision entering training, and use source-folded auxiliary solvers at feedback time to stop same-source pseudo-labels from being returned as reward. For practitioners this means self-generated curricula can keep scaling, but the evaluator's training history needs to be recorded; for researchers, the paper explicitly proposes extending exclusion to connected sources and measuring end-to-end efficiency.

Cross-fitting changes who supplies feedback, not what makes an answer true: shared pretraining, overlapping web evidence, semantically related sources, and an adaptive proposer can still induce correlated mistakes, which the paper lists as an open limitation. A lower false-agreement mass on accepted questions can also result from rejecting difficult tasks rather than improving learning, so the paper stresses that coverage, task difficulty, fixed-probe performance, and downstream capability must accompany that number. On cost, CrossFit adds 72%/79% over Dr. Zero and MSV about 90%, with the combination at 2.6/2.7 times the budget; the half-budget control lowers this to 36%/40% with nearly the same replay false agreement, but lower end-to-end cost or robustness to connected sources is not yet established. In addition, the audit uses gpt-6-astra/high as judge, unsupported cases remain unresolved and are included in coverage accounting, and automated judges can have systematic biases, which is why the paper emphasizes blinded evidence gathering and human validation.