Molecular traps expose retrosynthesis blind spots: top-20 error rates reach 100% on three selectivity problems
Synopsis
The study introduces "trap molecules," an adversarial evaluation idea in which an AI agent uses reaction SMARTS templates and USPTO reaction records to deliberately design molecules carrying sensitive groups, near-duplicate reactive sites or bulky groups that lure retrosynthesis algorithms into characteristic errors. On 100 chemoselectivity, 100 regioselectivity and 27 steric trap molecules, three algorithms built on different principles, UAlign, SimpRetro and RetroChimera, all showed clear weaknesses in their top-20 predictions, with overall problem rates of 66%–77%, 85%–93% and 92.6%–100% for the three problem types.
Interpretation
Evaluation of retrosynthesis algorithms moves from whether the answer appears in the top-k to whether the predictions themselves are chemically feasible, using trap molecules built to provoke characteristic errors. Conventional top-k accuracy only checks whether the recorded answer is among the predictions, not whether the other predictions are correct; solve rate only asks whether a target can be broken down to purchasable materials within limited steps and says little about route quality. On 100 chemoselectivity trap molecules, the pre-set matching test hit 51 for UAlign, 58 for SimpRetro and 56 for RetroChimera; adding agent-based judgment and manual confirmation raised these to 66, 77 and 71.
Trap molecules are generated by an AI agent following chemical design intent rather than selected as hard cases from existing data. Chemoselectivity and steric traps are guided by 100 common reaction SMARTS templates, while regioselectivity traps modify specific reaction SMILES records from USPTO. Regioselectivity traps require substrates with two or more reaction centers in similar but non-equivalent environments, for example two ester groups on a benzene ring distinguished only by a methyl versus an ethyl group; the 27 steric traps introduce bulky groups such as tert-butyl around the reaction center.
Agent-based judgment catches errors that pre-set matching misses, showing that rule-based preset testing needs a complement. Pre-set matching only compares against predefined erroneous reactions, so it can only verify whether an algorithm makes common, anticipated errors and may underestimate the problems. Agent-based judgment plus manual confirmation added 15, 19 and 15 hits for the three algorithms on chemoselectivity, 36, 40 and 24 on regioselectivity, and 4, 0 and 2 on steric hindrance.
Perspective
The evaluation covers three retrosynthesis algorithms with different design principles, UAlign, SimpRetro and RetroChimera, taking top-20 single-step predictions on chemoselectivity, regioselectivity and steric trap molecules; the trap sets contain 100, 100 and 27 molecules, and judgment combines pre-set matching, agent-based judgment and manual confirmation.
Predicted reactions do not specify complete reaction conditions, so chemoselectivity is judged mainly against commonly used conditions, which may introduce error; regioselectivity is assessed assuming the predicted reaction can proceed; some steric trap molecules may have no efficient synthetic method at all, and requiring a model to output predictions raises the probability of problematic cases; agent-based judgment is currently limited by cost and computational speed.