Paper2Bench turns published papers into auditable exams: the best agents match or beat every baseline on 80% of tasks
Spatial transcriptomics alignment lacked an auditable way to evaluate AI agents, so Paper2Bench turned two published alignment benchmark studies into 121 executable tasks, had agents solve them in containers with ground truth withheld, and re-ran 16 published baselines under one scorer. Across the 121 tasks, the best agents (GPT-6-Astra and Gemini-4-Argon) matched or outperformed all baselines on 80% of tasks; distilling agent runs into a standalone program beat 87% of baselines on held-out tasks.