# CyberGym

Automatically generated security patches can stop the crash and pass existing tests while quietly changing what the program does. DeepDiff treats the original vulnerable program as the behavioral reference and compares observable behavior on inputs that do not trigger the crash, returning the differing input as concrete evidence. On 385 candidate patches drawn from CyberGym, this regression-freedom check raised recall for functionally divergent patches from 6% under non-differential execution to 64%, at the cost of flagging 18% of developer patches as divergent.

Automatically generated security patches can stop the crash and pass existing tests while quietly changing what the program does. DeepDiff treats the original vulnerable program as the behavioral reference and compares observable behavior on inputs that do not trigger the crash, returning the differing input as concrete evidence. On 385 candidate patches drawn from CyberGym, this regression-freedom check raised recall for functionally divergent patches from 6% under non-differential execution to 64%, at the cost of flagging 18% of developer patches as divergent.

A multi-axis open-coding study of 200 CyberGym traces found that code localization and understanding (43.1%) plus vulnerability reasoning and trigger design (17.3%) account for 60.4% of tokens and that only 24.4% of matched comparisons preserve success at lower total cost; the authors then present AVRI, an interface built on a persistent Bidirectional Evidence Trace (BET), which on 20 evaluation tasks reduces total cost by 18.0% for Codex and 23.7% for OpenCode while preserving success and improving or maintaining recall.

A multi-axis open-coding study of 200 CyberGym traces found that code localization and understanding (43.1%) plus vulnerability reasoning and trigger design (17.3%) account for 60.4% of tokens and that only 24.4% of matched comparisons preserve success at lower total cost; the authors then present AVRI, an interface built on a persistent Bidirectional Evidence Trace (BET), which on 20 evaluation tasks reduces total cost by 18.0% for Codex and 23.7% for OpenCode while preserving success and improving or maintaining recall.