Agentic data synthesis stops just piling up samples: feedback rewrites the synthesis space, lifting task success from 64.5% to 77.8%
Synopsis
Agentic training-data synthesis has a new shape: EvoSyn turns learner feedback into a revisable synthesis space, so what gets generated follows the task agent's weak spots. Earlier methods used environment feedback, data validation, or learner failures, but observed weaknesses did not directly say what the next round should generate, nor whether existing synthesis dimensions could represent them. On τ-Bench and CAR-Bench with Opus 4.5 as the synthesis model, average task success rises from 64.5% for the strongest synthesis baseline to 77.8%, reaching 70.9% after the first round, with multi-round evolution adding further gains.
Interpretation
The synthesis process itself becomes a revisable object, rather than a fixed set of dimensions that emits tasks in bulk. Earlier synthesis methods used feedback to build executable tasks, validate or repair samples, or generate harder problems from failures, while the synthesis dimensions stayed fixed. On τ-Bench and CAR-Bench with Opus 4.5 as the synthesis model, EvoSyn-Final-SFT reaches 77.8% average task success, above Monolithic, Fixed-State, R-Zero-style, and AutoPlay-style synthesis variants, with EvoSyn-R1 at 70.9% after the first round.
The system maps agent performance, coverage, and failure diagnostics back onto the synthesis space, then decides whether to reallocate generation budget or add a controllable factor. Feedback no longer only filters or repairs individual samples; it decides what the synthesis process should change next. On CAR-Bench, removing learning feedback drops success from 0.713 to 0.660 and trajectory diversity from 0.769 to 0.538; removing the Reflector drops factor coverage from 0.893 to 0.803; removing the Designer drops success to 0.540.
Newly added factors act as real generation controls rather than post-hoc labels. A no-feedback control proposed six plausible dimensions but recovered none of the write- and read-volume factors actually adopted. The feedback-conditioned Reflector proposed two factors and both were adopted, while the no-feedback control recovered 0 of 6; in R2 synthesis, requested and realized num_write values match exactly at 99.6% to 100.0% across domains and synthesis backends.
Perspective
The report covers agentic data synthesis in interactive tool-use environments, evaluated on the Airline, Retail, and Telecom domains of τ-Bench and on CAR-Bench Base, with task success rate from the environment evaluator as the metric. Synthesis backends include Claude Opus 4.5, Qwen3-30B, and Qwen3-235B; the default task agent is Qwen3-8B, with 14B and 30B additionally examined. Synthesis runs for at most three rounds, and the Reflector introduces at most one new factor per round.
Multi-round gains are described as observed success-rate changes, and the authors state that the marginal learning-gain analysis across rounds is descriptive rather than a causal estimate of per-example utility. Factor controllability is verified for num_write, a directly measurable factor, while other newly added factors are not given the same per-factor evidence.