One corrupted detector out of four drops LLM investigators to 0.50–0.72 accuracy: CAV rechecks claims against evidence the tools cannot touch

When security operations centers put LLM agents into alert triage, the agent rarely inspects raw traffic; it reasons over labels and confidences emitted by intrusion detection systems, classifiers and anomaly detectors, which implicitly treats those tools as trustworthy. Prior work on evidence conflict for LLMs concerns conflicts between a model's own knowledge and the context it reads, or between retrieved documents of unknown reliability; this work addresses inter-tool conflict, where the agent has essentially no prior about an individual flow and its verdict is decided entirely by how it reconciles the tools. On the test split of the corrected CIC-IDS2017, the four detectors have test accuracies of 0.99997, 0.99998, 0.813 and 0.993, and their family-level verdicts disagree on 0.67% of test flows.

Under corruption, unverified LLM investigators follow a corrupted majority and also abstain on honest disagreement among the tools. One corrupted detector out of four lowers naive accuracy to 0.64–0.72 for Qwen-7B and 0.50–0.54 for Llama-8B; with three corrupted detectors every unverified LLM falls to 0.13 or below. On 300 test flows where the honest detectors disagree, reliability-weighted voting, Dawid–Skene and CAV-rule are at least 0.99 accurate, while naive Llama-8B abstains on 63% of these flows with accuracy 0.37.

CAV sits between the detectors and the investigator: it ignores reported confidences, quantifies conflict with the variation ratio, verifies only conflicting flows, and checks them against raw features the tools cannot influence. Naive investigators treat tool counts as evidence, citing tool conflict in 94% of abstention rationales and the number of agreeing tools in 95% of rationales when following a corruption; CAV instead tests the leading candidates with a nearest-neighbour trust score that does not vote. With the prompt frozen before testing, CAV-LLM improves on the naive investigator of the same model in all 12 model–condition pairs on the 210 shared flows under exact McNemar tests; with three corrupted detectors Qwen-7B reaches 0.87–0.88 and Llama-8B reaches 0.84.

Verification moves trust from the tools to the check itself, so the accuracy of that channel must be measured in turn. Moving 5–30% of the trust score's reference labels to a random other family lowers kNN-alone accuracy to 0.90–0.59; with at most one corrupted tool CAV inherits the channel's errors, whereas adding the channel as a fifth voter is better. The ordering reverses with more corruption: at four corrupted tools CAV is ahead at every noise level, with CAV-LLM at 0.81–0.52 against 0.27–0.07 for the +kNN vote.

The results rest on the corrected CIC-IDS2017, a 2017 laboratory dataset, and the random split overstates deployed detector accuracy; three rare families are excluded and unseen families are not evaluated. Corruptions are synthetic and non-adaptive and change only tool outputs, not traffic-level evasion that perturbs raw features; the models evaluated are open-weight models up to 14B parameters with 4-bit quantisation.

When the independent channel is itself noisy and few tools are corrupted, CAV inherits the channel's errors, and adding the channel as an extra voter is better. For a novel attack the trust score can at best support no known family, which CAV-rule turns into an abstention. Random reference-label noise is only a first step towards targeted poisoning, larger models may weigh evidence differently, and trust-related scores drift even across releases of one model line.