AI designing molecular cloning experiments: top model solves 30.6% on one try, 72% within ten
Synopsis
Molecular cloning demands end-to-end reasoning, from picking starting materials and designing sequences to planning a series of reactions, while models have been reliable only on individual steps. BenchBench-Cloning tests that end-to-end ability with 65 cloning tasks drawn from real experiments, split into protocol development and construct design. GPT-6 Astra solved 30.6% of tasks on a single attempt, ahead of GPT-6.1 Sol at 22.2% and Claude Opus 5.5 at 20.5%; the same top model solved 72% of protocol-development tasks within ten attempts, against 27% on one.
Interpretation
A benchmark built to test whether AI models can design complete molecular cloning experiments is now public, moving evaluation from single steps to end-to-end experiment planning. Models could already handle individual steps such as analyzing DNA sequences, but designing a whole experiment requires end-to-end biological reasoning. The benchmark holds 65 cloning tasks built with scientific experts and drawn from real experiments, in two types: protocol development and construct design.
Scores separate the models: GPT-6 Astra leads at 30.6%, followed by GPT-6.1 Sol at 22.2% and Claude Opus 5.5 at 20.5%, with Kimi K3 at 8.8%. The spread makes visible the gap between frontier and open-weight models on the same task. The scores come from evaluation on those 65 tasks, where the best result stays below one third of all tasks.
Failures cluster where correct individual decisions must be combined: models pick some right DNA constructs but omit a critical experimental control, design primers that target the correct sequence but cannot work together in a reaction, and plan the right reactions but fail to isolate the desired product. A correct single step does not make the whole experimental chain work. These failure modes recur across the evaluation and point to weak long multi-step reasoning.
Repeated attempts raise the ceiling substantially: the top model solves 72% of protocol-development tasks within ten attempts, against 27% on a single attempt. The capability is emerging, but single-attempt reliability has not caught up. The comparison is the same model on the same task type at one attempt versus ten.
Perspective
The evaluation covers 65 molecular cloning tasks drawn from real experiments, split into protocol development and construct design, and its conclusions are limited to this task set and the models tested. The releasing team says the BenchBench suite will expand to structuring messy scientific data, writing scientific reports, and evaluating biosecurity risks.
The releasing team states that the capability is emerging but the reliability is not there yet, and the gain from ten attempts shows a single result does not indicate stable use; with 65 tasks, the ranking reflects performance on this task set.