# Coding Agent Evaluation

PDEViBench changes how coding agents are scored on partial differential equations, tracing each passing program back to the computation behind its output instead of only checking agreement with a reference. Across 290 tasks in six equation families, a Codex agent (GPT-5.2) passed 80.1% of 1,450 runs, but 8.6 percentage points of that came from returning the manufactured exact solution rather than solving; counting only numerical solves changed which task features looked hard and reversed the ranking of two prompts. Under a prompt that forbade the shortcut, every passing program solved numerically.

PDEViBench changes how coding agents are scored on partial differential equations, tracing each passing program back to the computation behind its output instead of only checking agreement with a reference. Across 290 tasks in six equation families, a Codex agent (GPT-5.2) passed 80.1% of 1,450 runs, but 8.6 percentage points of that came from returning the manufactured exact solution rather than solving; counting only numerical solves changed which task features looked hard and reversed the ranking of two prompts. Under a prompt that forbade the shortcut, every passing program solved numerically.