Skip to main content
Back to timeline
arXivSource publication:

Scientific agents stop leaning on the last round: Sage-Taiyi keeps what it learns mid-search and tops Taiyi-Bench at 54.0

Synopsis

Sage-Taiyi splits scientific discovery into two levels: an object level that searches over a target tree and an idea graph, and a meta level that keeps a revisable cognitive state recording which judgments have evidence and which premises remain untested, then uses it to decide what to investigate next and which method to apply. On Taiyi-Bench, 265 tasks drawn from 50 papers across ten disciplines, it scores 54.0 overall against 47.9 for the strongest baseline, AI Scientist-v2, ranking first on all eight metrics. Five expert-guided collaborations in chemistry, materials and engineering carried some proposals to experimental or computational validation, including a cobalt-catalysed carbonylation model reaction raised from trace amounts to 84% yield.

Source-provided article image: Sage-Taiyi: A Metacognitive Architecture for Scientific Discovery
arXiv

Interpretation

Scientific discovery agents are beginning to treat the understanding built during a search as reusable grounds for decisions, rather than feeding only the latest round's results into the next. Earlier iterative scientific agents advanced mainly on feedback from the current round, so earlier failures, unresolved conflicts and dependencies shared across branches could be overlooked, and search could over-respond to recent outcomes. Across the 265 Taiyi-Bench tasks, Sage-Taiyi scores 54.0 overall against 47.9 for AI Scientist-v2; in pairwise comparisons it reaches an Elo of 1322.5 and wins 66.3% of comparisons against AI Scientist-v2.

The system stores its understanding of the problem separately from its command of research methods, so the former is revised as evidence arrives and the latter is reused across tasks. The object level organises targets and subtargets in a target tree, records ideas and their lineage in an idea graph, and keeps experimental results, literature knowledge and expert input with their sources and conditions in an evidence state. The meta level's cognitive state retains key findings, competing explanations, necessary assumptions and unresolved questions; a cognitive repertoire holds ten methodological heuristics that organise research paths plus cognitive operators reusable across workflows, and control selects targets and methods from the current state.

Monitoring and control form the loop between the two levels, bringing cross-branch issues such as shared premises and unresolved conflicts into the next decision. Monitoring reads the current cognitive state together with relevant judgments and evidence and assesses what changes imply for further search; control then decides whether to continue, revise, redirect or stop a branch. In a communication-engineering case, monitoring noted that several ideas shared a dependence on a passive reference, and control redirected attention to using the residual carrier as a seed for an actively regenerated reference, selecting operators that combined reference regeneration, sideband reflection and carrier-to-signal power ratio selection into one receiver candidate.

Perspective

The demonstrated scope is ideation evaluation on Taiyi-Bench, the AgentIdeaBench and IG-Arena benchmarks, and five collaborations in metal catalysis, photochemistry, metamaterials, alloy design and compressor engineering where domain experts took part in selecting and validating ideas; the metamaterials and compressor results come from simulation, and the alloy reports a mean tensile strength of 342 MPa across repeated experiments.

Reusing the cognitive repertoire across domains is listed as future work, and current results come from the benchmarks and five collaborations above; the metamaterial filter's 6.2% stopband and 94.5% passband average transmissions are simulated, and the alloy's 342 MPa is a mean tensile strength across repeated experiments.