HPC operations reports stop trusting retrieval alone: REPORT-OPS gates evidence before any claim ships

The trustworthiness of an operations report is split into two separately testable parts: the model interprets and writes, while explicit policy and automated checks rule on whether the evidence is fit. Earlier retrieval-based question answering and tool-using systems could fetch relevant records, but did not generally define which authority, semantic and temporal conditions make a conclusion usable. Across 68 queries, full REPORT-OPS reached 68.63% Pass@80 against 7.02% for the strongest retrieval baseline, with significant differences under both paired tests.

The system completes a request in one of three ways: answer when evidence supports the claim, qualify when only part is supported, and block when the central conclusion would need unsupported inference. The ungated REPORT-OPS variant reached 70.42% Pass@80, above the full system's 68.63%, and the difference was not statistically significant. The gate's main benefit is claim-support precision, which rose from 40.58% to 90.06% and proved most useful when evidence was missing or came from an inappropriate source.

Explanatory requests are handled as hypothesis testing over encoded explanation families rather than open-ended causal discovery. Converting raw evidence into typed records raised explanation accuracy only from 30.53% to 33.95%, and neither configuration produced passing explanatory reports. Adding author-seeded workload relations brought Pass@80 to 33.97% and explanation accuracy to 51.37%; full claim verification raised these to 41.02% and 55.54%, though the gain over relations alone was not significant.

Policy improvement is treated as a controlled release: a critic model proposes localized patches, a human approves or rejects without rewriting, and automated tests decide promotion. Localized updates corrected 91.17% of targeted failures over five rounds and improved related held-out cases by 9.33 percentage points; without the regression gate, a 6.06% regression break rate was observed. Of 15 agent-round opportunities, 10 candidates passed the full gate and 5 retained the previous policy, and the gated strategy showed a final observed regression break rate of zero.

The evaluation runs on a frozen, sanitized corpus from one HPC facility, with 68 queries expanded from 50 base scenarios covering complete, missing, temporally invalid, authority-swapped, contradictory and tool-failure evidence conditions; explanatory conclusions cover only 13 queries and encoded explanation families. The method addresses facility operators and engineers who need auditable operations reports, and cross-facility, non-HPC and cross-agent policy transfer were not evaluated.

The authors note that the benchmark, reference rubrics and initial policy come from the same corpus, so domain expertise may shape both expected answers and system policy; annotations were prepared by the authors and reviewed by a second author, not by external HPC operators. The two judge models agreed on more than 96% of Pass@80 decisions but differed by roughly ten points on average in continuous scores. The gated strategy's zero regression break rate is an observed outcome of these five rounds, while the gate itself permits up to 10% breakage and does not guarantee that future updates avoid regressions.