Skip to main content
Back to timeline
arXivSource publication:

A language-model agent writes its own physics-residual detector: BATADAL test-attack F1 rises from 0.504 to 0.878

Synopsis

For runtime attack detection in cyber-physical systems, PIA has a large language model agent write physical residuals such as pump characteristics, tank mass balances and controller logic from a plant description and normal operating data, then test each candidate on an evaluation workbench before deployment. On the BATADAL water-distribution attack benchmark's test attacks, PIA with Claude Opus 5.5 reaches an F1 of 0.878 against 0.504 for the strongest of six statistical and deep baselines; ablations show that denying the agent data testing costs up to 0.28 F1.

Source-provided article image: PIA: Agentic Physics-Informed Runtime Anomaly Detection
arXiv

Interpretation

Runtime detectors no longer have to be black boxes learned from normal data, nor physical relations derived by hand for every plant; PIA has a language model agent write the detector itself as a set of physical residuals, each stating a relation that should hold in normal operation. Learned detectors such as autoencoders and recurrent forecasters model only the joint distribution of all sensors without knowing which relations are physically binding, so stealthy manipulations that keep every signal inside its normal range, such as a replayed tank level or an offset pressure reading, pass unnoticed and an alarm gives no indication of where the problem lies; hand-built physical residuals avoid both issues but require an engineer to derive, parameterize and tune every residual for the specific plant. On the test attacks of the BATADAL water-distribution attack benchmark, PIA with Claude Opus 5.5 reaches an F1 of 0.878 against 0.504 for PCA, the strongest of six statistical and deep baselines; the gain comes from recall, which grows from 0.349 to 0.874, because the baselines catch the onset of most attacks but lose them once concealment takes effect.

The detector is designed once by the agent before deployment and runs without the model, evaluating a small set of fitted arithmetic expressions, so its behaviour can be audited line by line and each alarm points to a component. The agent receives a short natural language description of the plant and a workbench over normal data, proposes residuals in a restricted expression language, observes how each candidate fits held-out normal data, its false alarm rate and its reaction to synthetic sensor manipulations, and revises the set until the detector is quiet on normal data and covers every sensor and actuator. An Opus design session issued 52 tool calls on average, consumed 364 thousand input and 23 thousand output tokens, and finished in 194 seconds; the deployed detector holds about twenty fitted expressions, is fitted in 0.13 seconds, and scores the three-month test period in 33 milliseconds on a CPU without a GPU and without calls to the model.

Perspective

The evaluation covers the C-Town water distribution network in BATADAL: 43 signals, one year of normal operation, Dataset 2 with seven attacks and a three-month test set with seven attacks, with all detectors and agents using only the normal data and labels taken from the organizers' published attack schedules. The detector targets plant operators who need online monitoring at the supervisory sampling rate; design happens once, and the language model is not called at run time.

Attack 14 alters the T4 level on its way to a controller while the reported level stays in its normal band, and all detectors, PIA included, flag it only around a single transient. The authors state that how residual programs should be revised when plant configuration, operating conditions or sensor reliability change remains an open question; such revision would require validating newly proposed relations against normal operating data while preserving interpretability and low runtime cost, and lies outside the present evaluation.