Production AI optimization stops reading only failures: CoCoEvolve mines the whole run history for 92.8% on DABStep hard tasks

Automatic optimization of production AI systems no longer looks only at a single candidate on the search frontier; it folds the whole evolutionary history and live probes of the running system into the decision about what to change. Earlier optimization methods picked a candidate from the search frontier and mutated it in one LLM pass based only on its failures, an approach that can stall on production systems where isolated failures reveal little about complex, hidden interactions. On DABStep, CoCoEvolve tops the public leaderboard with 92.8% held-out hard accuracy, compared with 78.4% for Meta-Harness and 30.4% for GEPA under matched conditions.

The mutation is carried out by an agent that first selectively retrieves historical evidence, then probes the running system, and only after verifying proposes a change. This mechanism is called meta-reflection, and the agentic mutator behind it is named Cortex Code; it no longer acts on a single failure read. Matched ablations confirm that each component of CoCoEvolve contributes to the gains; it outperforms every evaluated baseline across six LLM data pipeline workloads and lifts test-set solve rate on data-eng-bench from 59.1% to 77.4%, past the best baseline of 73.6%.

This report draws on summary-level material covering three classes of production AI systems: data agents, LLM-powered data pipelines, and data engineering workflows. Reported results include topping the DABStep public leaderboard, outperforming every evaluated baseline across six data pipeline workloads, lifting the data-eng-bench test-set solve rate, and exceeding GEPA by 11 percentage points on HotpotQA. The summary states no next experiment or deployment recommendation.

The summary says the method remains competitive with specialized optimizers on established mathematical and systems benchmarks without giving those numbers. Matched ablations confirm each component contributes, but the summary does not list the ablation scale or settings. These qualifications come from summary-level material.