# CWE-bench v2

Collinear AI released CWE-bench v2, a public set of 100 tasks in which an agent gets a checkout of a real open-source repository and must find and fix a security weakness without breaking existing functionality, covering 91 CWEs. Scoring is fully programmatic and reproducible, and a held-out set tracks tasks no model has seen. Among 11 evaluated models, Gemini 4 Argon leads at 67.8% pass@1 and Claude Opus 5.5 follows at 67.3%; 95 tasks are solved by at least one model, yet the leader fails 19% of the set on every attempt.

Collinear AI released CWE-bench v2, a public set of 100 tasks in which an agent gets a checkout of a real open-source repository and must find and fix a security weakness without breaking existing functionality, covering 91 CWEs. Scoring is fully programmatic and reproducible, and a held-out set tracks tasks no model has seen. Among 11 evaluated models, Gemini 4 Argon leads at 67.8% pass@1 and Claude Opus 5.5 follows at 67.3%; 95 tasks are solved by at least one model, yet the leader fails 19% of the set on every attempt.