Skip to main content
Back to timeline
arXivSource publication:

Scientific agents stop working alone: a shared research state turns failed experiments into the next agent's starting point

Synopsis

Self-improvement in scientific agent swarms no longer rests on one agent's repeated trial and error. Previously, a failed experiment could not show whether the hypothesis or its implementation was at fault, and a program without overall gains could still hide a component another agent needed. SwarmRSI places hypotheses, programs and experimental evidence into a shared graph as inheritable research states, uses other agents' feedback to decide whether to revise a hypothesis or repair a program, retests retained hypotheses on other agents' programs, and combines complementary components. It reaches an average of 76.2 on OpenProblems and matches published SOTA on 44.4% of NatureBench tasks, the highest rate among compared methods, with medals on 81.8% of MLE-Bench Lite tasks.

AI-generated editorial illustration: SwarmRSI: Recursive Self-Improvement through Evolving Scientific Agent Swarm

Interpretation

Self-improvement in scientific agent swarms is recast as evolving inheritable research states rather than improving one agent's code, policy or workflow. Each agent assembles hypotheses, programs, experimental evidence and unresolved needs from a shared graph to plan its next step, so one experiment changes the state later agents start from. Program evolution previously refined executable candidates through execution feedback alone, and hypothesis evolution improved directions through iterative generation and evaluation; existing systems linked hypotheses to programs and evidence in a persistent hypothesis tree or evaluated competing failure explanations, but did not let one agent's feedback and intermediate results rewrite another agent's starting point. Across all 90 NatureBench tasks, removing the shared graph lowers Match-SOTA by 7.8 percentage points and removing adaptive hypothesis/program selection lowers it by 8.9 points, with paired 95% confidence intervals excluding zero for the mean drop of all five ablation variants.

Feedback-Guided Swarm Mutation uses other agents' experimental feedback to decide whether to revise a hypothesis or repair its program: differences across instances point to a revised research direction, while execution failures or unrealized interventions point to implementation repair. Complementary Swarm Recombination reuses programs whose overall scores did not improve. A failed experiment previously left unclear whether the research direction or its implementation needed revision, and a program that performed poorly overall was easily discarded along with the useful components inside it. On the six OpenProblems single-cell tasks, SwarmRSI reaches an average of 74.2 with DeepSeek-V4-Pro against 67.1 for TusoAI, and 76.2 with Gemini 3.1 Pro Preview, the highest among compared methods. On NatureBench with the same backbone, its Match-SOTA rate exceeds Arbor by 7.8 points and AIDE by 16.7 points, with paired bootstrap 95% confidence intervals of [2.2, 13.3] and [8.9, 24.4].

Recombination retests retained hypotheses on other agents' programs and combines program components that are stronger on some instances. Rebase keeps the hypothesis content and swaps only the implementation base, while crossover picks donors by their observed per-instance advantage, taking donor components on the instances they win and keeping the parent's components elsewhere. Earlier approaches either evolved within a single program lineage or excluded programs without overall gains from later search, leaving cross-agent reuse of hypotheses and components without a fixed mechanism. In one DeepSeek-V4-Pro seed covering all 90 NatureBench tasks, about 77% of final best programs descend from at least one rebase, crossover or feedback-triggered update; those updates comprised 296 attempts and produced 84 new bests, a 28.4% new-best rate against 21.9% overall.

Evolved programs were reproduced by biologists and used in follow-up analyses, not left at benchmark scores. Benchmark scores alone do not show whether a program is usable in real data analysis, and earlier agent systems rarely reported independent reproduction of generated programs by domain experts. On intestinal data, the reproduced program gives covariate-adjusted Spearman correlations of 0.77 and 0.69 for MHC-II local coherence in the two helminth day-10 sources, against 0.65 and 0.40 for RNA PCA; on spleen data, the program raises the CytoCommunity task metric from 0.045 to 0.102, after which experts group B-cell neighbourhoods by local cell composition.

Perspective

The method applies to research tasks where candidate programs can be executed and scored, and evaluation covers three benchmark families, OpenProblems, NatureBench and MLE-Bench Lite, under matched budgets against program-evolution, hypothesis-evolution and general coding agents. Cross-task evidence comes from six source tasks, 12 held-out tasks and one backbone, where three generations of inherited state raise held-out mean from 12.01 to 21.89 and Match-SOTA from 41.7% to 86.1%. The biological cases analyse existing datasets, and the programs were reproduced by biologists for follow-up analyses of intestinal and splenic immunity.

Execution feedback does not always reveal whether the intended intervention was faithfully implemented, and performance gains alone do not establish its scientific mechanism. Rebase selects program bases by matching recorded implementation needs to program mechanism labels, a heuristic that does not measure what a program can do; crossover applies only when a benchmark reports scores per instance; hypothesis/program selection adapts discounted UCB without inheriting its guarantees, and its benefit is established only empirically. The human failure analysis characterizes 644 failed attempts from 35 task trajectories, and cross-task generations are confounded with the number of source tasks seen. The evaluation does not separately assess literature synthesis, idea quality or peer-review quality, nor does it test legal reasoning, zero-shot entity extraction, conversational recommendation or shopping research.