Rewrite the same commit, and frontier agents lose the bug: transformations mined from real development cut vulnerability-discovery success from 63.3% to 36.8%

The same code, written differently, can hide a vulnerability that a frontier agent previously found. Vulnerability-analysis evaluations typically target a single version of the software and attribute performance differences to vulnerability characteristics and agent capability. In CyberGym, agent success fluctuates across adjacent commits in one repository that carry the same functionality and the same vulnerability, with runs either exhausting the budget or producing a proof-of-concept that cannot trigger the vulnerability.

The approach distills recurring changes from real development into reusable transformation skills, then instantiates them against the target code's context. Earlier robustness evaluations applied fixed local obfuscations such as variable renaming, while-to-for rewrites, and unreachable-code injection. Mining C and C++ commits from CommitPackFT yields 48 transformation categories, and the evaluation uses the 21 most frequent, which cover 84% of commits in the dataset.

Transformations can be composed, and composition suppresses the agent further. A single transformation already lowered success, but transformations from different families had not been combined. For GPT-5.6 Sol with Codex, composing S2, R3, and P1 into pairs and a triple lowers the mean success rate from 34.4% for the three constituents alone to 26.7% for pairs and 20.0% for the triple.

Letting the agent rewrite the code before analysis does not recover the gap. Synthetic obfuscations are easily reversed once the agent knows which one was applied. Told which transformation was applied, GPT-5.6 Sol with Codex recovers 25.9 percentage points on synthetic obfuscations on average but only 3.5 points on natural transformations, remaining 30.1 points below its success on the original code.

The work addresses software maintainers who need to resist automated vulnerability scanning, and it demonstrates effects on C and C++ codebases under CyberGym's level 0 setting. The evaluation covers six models paired with three agent harnesses, and each transformed variant is rebuilt, run against the codebase's own test suites, and checked by replaying the ground-truth proof-of-concept so that functionality and the vulnerability are preserved.

Transformations are extracted and applied by code agents, which limits interpretability, and mining draws on a single dataset of roughly 700K commits, with the authors expecting more transformations from larger C and C++ codebases. Evaluation concentrates on CyberGym's hardest level 0 setting, the authors call for expanding to a broader set of settings, and they note that stronger guarantees of semantics preservation would improve deployability in realistic development.