Production AI optimization stops reading only failures: CoCoEvolve hits 92.8% hard accuracy on DABStep
Synopsis
Evolutionary optimization for production AI systems no longer picks one candidate from the search frontier and rewrites it from its failures alone. CoCoEvolve has an agentic mutator retrieve evidence from the entire evolutionary history and probe the live system before proposing changes. On DABStep it reaches 92.8% held-out hard accuracy, against 78.4% for Meta-Harness and 30.4% for GEPA under matched conditions; on data-eng-bench it lifts test-set solve rate from 59.1% to 77.4%, past the 73.6% best baseline.
Interpretation
Automatic optimization of production AI systems no longer looks only at a single candidate on the search frontier; it treats the whole evolutionary history as available evidence. The prior approach selected a candidate from the search frontier and mutated it in one LLM pass based only on its failures; when a system hides complex interactions, isolated failures reveal little and optimization can stall. On DABStep, held-out hard accuracy is 92.8% under matched conditions, compared with 78.4% for Meta-Harness and 30.4% for GEPA.
Before any mutation is proposed, the optimizer probes the live system, then verifies and proposes the change. The mechanism is called meta-reflection, and the mutator is an agentic component, Cortex Code, which selectively retrieves evidence from the entire evolutionary history. It was applied to three classes of production AI systems: data agents, LLM-powered data pipelines, and data engineering workflows; it outperforms every evaluated baseline across six LLM data pipeline workloads, and on data-eng-bench lifts test-set solve rate from 59.1% to 77.4%, above the 73.6% best baseline.
On established mathematical and systems benchmarks it remains competitive with specialized optimizers. On HotpotQA its accuracy is 11 percentage points higher than GEPA. Matched ablations confirm that each component of CoCoEvolve contributes to these gains.
Perspective
This report covers summary-level information, not the full experimental detail of the paper. The demonstrated scope is three classes of production AI systems: data agents, LLM-powered data pipelines, and data engineering workflows, with comparisons on DABStep, six data pipeline workloads, data-eng-bench and HotpotQA, plus matched ablations. It suits researchers and engineers working on automatic optimization of LLM systems and data engineering pipelines.
The summary states competitiveness with specialized optimizers on established mathematical and systems benchmarks without giving those numbers. The gains from meta-reflection are supported by matched ablations, but the summary does not describe their scale or setup.