# Claim Verification

Operational reporting for high-performance computing facilities has to assemble monitoring, scheduler, incident and storage records into a traceable conclusion, and a retrieved record is not automatically fit to support a specific claim. REPORT-OPS lets a large language model interpret requests, plan evidence collection and draft reports, while author-seeded policies and automated checks decide whether the evidence qualifies, then answer, qualify the conclusion or withhold it. On 68 sanitized operational queries, the full system reached 68.63% Pass@80 against 7.02% for the strongest retrieval baseline, and evidence gating raised claim-support precision from 40.58% to 90.06%.

Operational reporting for high-performance computing facilities has to assemble monitoring, scheduler, incident and storage records into a traceable conclusion, and a retrieved record is not automatically fit to support a specific claim. REPORT-OPS lets a large language model interpret requests, plan evidence collection and draft reports, while author-seeded policies and automated checks decide whether the evidence qualifies, then answer, qualify the conclusion or withhold it. On 68 sanitized operational queries, the full system reached 68.63% Pass@80 against 7.02% for the strongest retrieval baseline, and evidence gating raised claim-support precision from 40.58% to 90.06%.

The work proposes two enforceable contracts for LLM agents over databases: a plan contract that statically checks plan structure, result-field references, and grounded identifiers before execution, and an evidence contract that records belief state, completeness, and provenance so claims can be verified; in TGMS, result-field checking turns invalid references into repairable rejections, completeness propagation detects truncated-count failures in all 15 controlled cases, and claim gating leaves none of 199 emitted answers with an unsupported gated claim at the cost of 21 fewer answers.

The work proposes two enforceable contracts for LLM agents over databases: a plan contract that statically checks plan structure, result-field references, and grounded identifiers before execution, and an evidence contract that records belief state, completeness, and provenance so claims can be verified; in TGMS, result-field checking turns invalid references into repairable rejections, completeness propagation detects truncated-count failures in all 15 controlled cases, and claim gating leaves none of 199 emitted answers with an unsupported gated claim at the cost of 21 fewer answers.

Using a Puppeteer crawler on AWS Lambda, the authors ran a 40-day longitudinal measurement (March 13–April 21, 2026) over 55,393 trending Google queries across 19 topical categories and found that AI Overviews activate on 13.7% of queries overall (64.7% for question-form queries), that cited domains are more credible than co-displayed first-page results yet 29.8% do not appear on the first page at all, that 11.0% of 98,020 atomic claims are unsupported by the cited pages with omission as the dominant failure mode, and that at least 50.6% of cited pages carry display advertising.

Using a Puppeteer crawler on AWS Lambda, the authors ran a 40-day longitudinal measurement (March 13–April 21, 2026) over 55,393 trending Google queries across 19 topical categories and found that AI Overviews activate on 13.7% of queries overall (64.7% for question-form queries), that cited domains are more credible than co-displayed first-page results yet 29.8% do not appear on the first page at all, that 11.0% of 98,020 atomic claims are unsupported by the cited pages with omission as the dominant failure mode, and that at least 50.6% of cited pages carry display advertising.