# Hallucination Detection

Chest X-ray vision-language models can write fluent reports that nonetheless assert findings absent from the image, and earlier fixes mostly retrained the generator through reinforcement learning, preference optimization or chain-of-thought supervision. CheXPoT instead works after generation: it decomposes a report into checkable predicates, tests each against a visual evidence table built from a chest X-ray classifier and Grad-CAM locations, and rewrites the report after removing predicates the image cannot support. On a 4,145-image held-out split, MedGemma-4B reports moved from 19.2% to 46.8% micro-average precision and from 23.0% to 49.4% micro-average F1, with significant precision gains on 8 of 14 CheXpert classes and no significant decreases in precision or F1.

Chest X-ray vision-language models can write fluent reports that nonetheless assert findings absent from the image, and earlier fixes mostly retrained the generator through reinforcement learning, preference optimization or chain-of-thought supervision. CheXPoT instead works after generation: it decomposes a report into checkable predicates, tests each against a visual evidence table built from a chest X-ray classifier and Grad-CAM locations, and rewrites the report after removing predicates the image cannot support. On a 4,145-image held-out split, MedGemma-4B reports moved from 19.2% to 46.8% micro-average precision and from 23.0% to 49.4% micro-average F1, with significant precision gains on 8 of 14 CheXpert classes and no significant decreases in precision or F1.

The work introduces an internal hidden-state framework for fine-grained, span-level hallucination detection that inspects layer-wise activation patterns to locate the onset and continuation tokens of hallucinations in large language model generations; experiments show substantial improvements in Precision-Recall AUC over random baselines despite extreme class imbalance, and it further proposes a cross-model detection framework in which one model observes the internal representations elicited by another model's generation, finding that an external observer can match or exceed the generator's self-detection of its own hallucination onsets, including when the observer is the smaller model.

The work introduces an internal hidden-state framework for fine-grained, span-level hallucination detection that inspects layer-wise activation patterns to locate the onset and continuation tokens of hallucinations in large language model generations; experiments show substantial improvements in Precision-Recall AUC over random baselines despite extreme class imbalance, and it further proposes a cross-model detection framework in which one model observes the internal representations elicited by another model's generation, finding that an external observer can match or exceed the generator's self-detection of its own hallucination onsets, including when the observer is the smaller model.

The study adapts the LLM Uncertainty Head framework to two Persian medical models built on Aya-Expanse-8B, Gaokerena-V and Gaokerena-R; it first observes on a 168-question Iranian medical entrance examination that Gaokerena-V has substantially lower five-run consistency than Aya-Expanse-8B while Gaokerena-R is comparable, then builds Persian claim-level hallucination datasets with 1,600 responses per backbone and trains lightweight claim-level heads on frozen backbone attention maps and token probabilities, obtaining held-out PR-AUCs of 0.4820 and 0.4652 (2.30 and 2.66 times their random baselines) and ROC-AUCs of 0.7852 and 0.7810, with no retrieval or repeated sampling at inference.

The study adapts the LLM Uncertainty Head framework to two Persian medical models built on Aya-Expanse-8B, Gaokerena-V and Gaokerena-R; it first observes on a 168-question Iranian medical entrance examination that Gaokerena-V has substantially lower five-run consistency than Aya-Expanse-8B while Gaokerena-R is comparable, then builds Persian claim-level hallucination datasets with 1,600 responses per backbone and trains lightweight claim-level heads on frozen backbone attention maps and token probabilities, obtaining held-out PR-AUCs of 0.4820 and 0.4652 (2.30 and 2.66 times their random baselines) and ROC-AUCs of 0.7852 and 0.7810, with no retrieval or repeated sampling at inference.