A clean sheet proves little on its own: how much a failure erases sets the sample cost of certifying agent risk
Synopsis
Dangerous-capability evaluations often use the milestone method: split a multi-step task into ordered milestones, run the agent a few times from prepared states, and record a clean sheet if every run fails. A new analysis partitions the task into checkpoint segments, encodes how much progress a failure erases through the reset rule, and shows that counting possible completion windows yields a bound on completion probability for any specified attempt budget. In rare-risk phase interiors, the minimum expected number of zero responses is governed by the mean length of active segments, and every asymptotically optimal audit must divide its responses equally among active milestones.
Interpretation
A clean sheet speaks about single attempts from prepared states, while a deployment risk statement concerns a campaign of many attempts; the bridge between them is how much progress a failure erases. The milestone method previously ran the agent a few times from prepared states and recorded an all-zero result, without saying what the same record implies under different reset rules. With four milestones and ten zero responses each, a target completion probability and a confidence level, an analytic bound covers up to 169 primitive attempts under full reset and a computer-assisted bound up to 185, but only up to 10 under full progress retention.
Once the task is cut into consecutive checkpoint segments, completing the chain requires an uninterrupted run through each segment, and counting where those completion windows can fall gives a certificate at a finite budget. The certificate holds for any specified budget and covers adaptive deployment processes whose success probabilities after every complete history are capped by the corresponding audit probabilities. The selected segments are fixed before the audit, each milestone is queried a fixed number of times, and certification requires all responses to be zero; unqueried milestones are unrestricted.
In a rare-risk phase interior each segment is either rarely completed within the budget (active) or nearly always completed (free), and the evidence is governed by the mean length of the active segments. The certificate attains that asymptotic constant, and every asymptotically optimal evaluator must divide its expected response count equally among active milestones in the limit, with a vanishing share on free ones. Where a single segment passes from active to free, each of its milestones receives a smaller limiting share than an active milestone, and a crossover-aware certificate attains the optimal constant.
Perspective
The results rest on the checkpoint-segment model and domination conditions: a failure erases only progress within the current segment, each deployment attempt consumes one unit of budget, and success probabilities after every complete history are capped by the audit probabilities. Finite certificates hold for any specified budget, while the asymptotic constants and allocation results are limited to rare-risk phase interiors and to a single crossover with a positive finite limit.
The transport assumption is not tested by the audit and can fail when prepared milestone states differ from those reached in deployment, when attempts share a latent cause, or when the agent improves across attempts. The model excludes alternative routes and skipped milestones, and reaching milestone states is not costed; complexity results concern the zero world only, finite-budget optimal allocations need not be uniform, and crossovers of several equal-length segments, fixed risk levels and nonzero safe parameters remain open.