Agents iterate their own data recipes: math and code selection strategies gain up to 13.34% under fixed training
Synopsis
Agents use experimental feedback to revise data-engineering strategies, and across four tasks—math and code data selection, math reward design, and math curriculum learning—the selected strategies gain up to 13.34% absolute over generic baselines on AIME25 Pass@32 under fixed training conditions.
Interpretation
Data-engineering strategies can be revised and combined by agents across experiments instead of being fixed once by people. Such strategies have typically been set by hand from a preset plan, without a mechanism that improves them from cumulative experimental feedback. Across four tasks—math and code data selection, math reward design, and math curriculum learning—strategies selected by in-loop scores gain up to 13.34% absolute on AIME25 Pass@32 under fixed training conditions.
Agents decide whether to pursue, revise, or combine research directions based on cumulative findings. The framework, Agentic Data Engineering (ADE), has agents use experimental feedback under fixed training and evaluation protocols. Strategies are selected by in-loop scores and evaluated on held-out benchmarks, where the average absolute gain across four held-out benchmarks is 5.85%.
In-loop scores rise over successive experiments, but held-out performance does not always follow. Held-out trajectories show that later in-loop gains do not always improve generalization. Comparisons of research organization and feedback access favor combining evidence analysis with cumulative findings.
Perspective
This report draws on summary-level material covering four tasks: math and code data selection, math reward design, and math curriculum learning, all under fixed training and evaluation protocols, with strategies selected by in-loop scores and evaluated on held-out benchmarks. It suits researchers and engineers interested in data engineering and AI4AI experimental workflows. The material states no next-step experiment plan.
The material is a summary and does not give full settings or statistical detail for each task. One reported point is that in-loop scores improve over successive experiments while held-out trajectories show that later in-loop gains do not always improve generalization. Comparisons of research organization and feedback access favor combining evidence analysis with cumulative findings, a conclusion limited to the reported tasks and protocols.