CADS scores an LLM proposer against the same screen with the LLM removed, and raw hit counts overstate its gain
Synopsis
In sequential biological screens, LLM-proposed candidates are usually credited by hit counts, but the re-selection policy and random fallback also produce hits on their own. CADS runs the full loop beside an otherwise identical proposer-removed loop, certifies each discovery set separately, and reports the paired difference; in DepMap-backed simulations the no-LLM loop alone cleared the standard feedback gate, and origin counts inflated Qwen2.5-7B's net contribution from near zero to apparently positive.
Interpretation
Measuring what an LLM proposer adds to a biological screen requires a control loop with the proposer removed and everything else held identical, not a count of hits attributed to the model. The usual practice credits the proposer with raw hit counts or origin attribution, yet the loop's re-selection policy and fallback also find hits, and LLM picks can displace discoveries the fallback would otherwise make. In simulations backed by DepMap 26Q1 ACH-000446, the no-LLM loop certified true discoveries under full feedback, with paired intervals against both absent and permuted feedback excluding zero, so the standard feedback gate passes with no LLM in the loop.
CADS turns each round's readout into a capped evidence increment, accumulates it across rounds as an anytime-valid e-process, and makes one terminal e-BH decision over the complete registered family to control the false discovery rate. The capped likelihood-ratio increment limits any single readout, the e-process permits adaptive sampling and optional stopping, and complete-family e-BH gives a terminal decision under arbitrary dependence; these statistical parts come from established bandit multiple-testing and robust e-value methods. At the frozen operating point, CADS certifies 245.7 true discoveries and 0.033 false discoveries on average with power 0.396; naive BH reaches power 0.816 but averages 300.8 false declarations at FDR 0.372, and uncapped LR e-BH gives FDR 0.251.
Origin attribution overstates the proposer's net contribution because slots the model fills displace hits the fallback would have found. Under the thin prompt Qwen certifies 2.95 discoveries per campaign, and counting LLM-originated versus fallback-originated discoveries makes the LLM look like the majority, but the LLM fills only 57.7 fresh slots while the fallback fills the rest and finds its own hits. With the LLM removed the fallback fills every slot and the net contribution falls to 0.30, so only the head-to-head pairing against the no-LLM loop gives the net gain.
Perspective
The measurement is demonstrated in sequential simulations backed by public reference effects including DepMap 26Q1 ACH-000446, covering one T-cell Perturb-seq reference, one PRISM reference and six DepMap 24Q4 cell-line references, not prospective wet-lab campaigns; readers should treat it as a measuring instrument for screening loops rather than a biological conclusion.
The guarantee depends on a pre-registered candidate family, a correctly specified one-sided null and a valid contamination bound, and misspecifying the null can invalidate control; Qwen's initial 20-seed net effect did not persist in a 50-seed matched-quota replication, no model's prompt effect excludes zero after Holm correction, and a fixed non-LLM prior remains stronger than all three models.