AI agents vote down their own run: a diamond-spin lab rejects a passing experiment, then earns the evidence
Synopsis
A laboratory of large-language-model agents, each holding a separate research role, asked how many calibration measurements are needed to predict a diamond nitrogen-vacancy spin ensemble's response to arbitrary phase-controlled microwave pulse sequences. The principal-investigator agent tightened a proposed bound of 91 calibrations to an exact minimum of 10; the first acquisition met all ten precommitted accuracy criteria but failed phase and reference controls and was ruled inconclusive, and after the agents revised the protocol without relaxing the criteria, the final measurement supported the prediction while two interpolation methods predicted zero.
Interpretation
An autonomous research system no longer runs as one model advancing step by step: proposal generation, theory, computational checking, experiment and review sit with separate large-language-model agents that can reject one another's work. Earlier automated laboratory platforms could generate hypotheses and test them in the lab, but could reach conclusions the evidence does not support. The principal-investigator agent rejected 75 proposals over roughly 30 hours and accepted none during the first 17 batches, citing previously established results, logical inconsistencies, insufficient significance and the absence of feasible experimental tests.
The research question came from rewriting a rejected proposal rather than from accepting an initial one. Repeatedly rejected directions accounted for 52 of 56 earlier fresh proposals. After the manager agent added those rejected mechanisms to the exclusion instructions, nine new proposals explored seven previously unrepresented research directions.
The theory turned the calibration count from a sufficient upper bound into an exact minimum. The original proposal established only a sufficient upper bound of 91 calibrations and did not determine the exact number required. The principal-investigator agent proved that 10 calibrations are both necessary and sufficient for phase-controlled sequences of at most a given number of pulses, independent of the spin-error distribution, and the computational agent independently reconstructed the response-space structure, confirming dimensions of 26 and 37 for sequences of up to five and six pulses.
Acceptance criteria were fixed before acquisition, the first run was ruled inconclusive on failed controls, and the final acquisition passed every criterion after the protocol changed. The first acquisition met all ten precommitted accuracy criteria but failed the global-phase and reference-pulse controls. The experimentalist traced the failure to signal drift during acquisition, with the signal rising 7.6% over the morning and a further 2.9% step at 11:48 a.m. when a laboratory member switched on the room lights; the final acquisition yielded 21,600 valid one-second records, and the four withheld targets matched the law's prediction while the interpolation alternatives predicted zero.
Perspective
The demonstration is limited to ensembles of nitrogen-vacancy centres in diamond at room temperature under phase-controlled microwave pulse sequences, with human researchers assembling the apparatus and setting voltage and power limits for the safety agent; the authors propose extending the approach to other quantum platforms and further experimental setups without demonstrating it.
The authors note limited diversity in the research ideas the agents generated, and new directions appeared only after the manager used the record of rejected proposals to revise the search strategy; extending to more experimental setups would require more reliable instrument diagnostics, coordination and safety mechanisms.