Across three 7B-scale models and three datasets, the hallucination signal is dominated by a single mean-shift direction, and simple L2 logistic regression reaches 0.952 AUROC while bounding or outperforming twelve architectural alternatives
This work characterizes the geometry of hidden-state probe signals for LLM hallucination detection across three 7B-scale models and three datasets in a paired-example paradigm, finding the signal overwhelmingly dominated by a single mean-shift component whose removal collapses detection to chance, with shrinkage linear discriminant analysis closing about 73% of the gap between 1D and full-dimensional classifiers, a simple L2-regularized logistic regression at 0.952 AUROC bounding or outperforming twelve controlled architectural alternatives, and LayerMix multi-layer aggregation matching oracle-layer performance without oracle access.