Jungseob Lee

This work characterizes the geometry of hidden-state probe signals for LLM hallucination detection across three 7B-scale models and three datasets in a paired-example paradigm, finding the signal overwhelmingly dominated by a single mean-shift component whose removal collapses detection to chance, with shrinkage linear discriminant analysis closing about 73% of the gap between 1D and full-dimensional classifiers, a simple L2-regularized logistic regression at 0.952 AUROC bounding or outperforming twelve controlled architectural alternatives, and LayerMix multi-layer aggregation matching oracle-layer performance without oracle access.

This work characterizes the geometry of hidden-state probe signals for LLM hallucination detection across three 7B-scale models and three datasets in a paired-example paradigm, finding the signal overwhelmingly dominated by a single mean-shift component whose removal collapses detection to chance, with shrinkage linear discriminant analysis closing about 73% of the gap between 1D and full-dimensional classifiers, a simple L2-regularized logistic regression at 0.952 AUROC bounding or outperforming twelve controlled architectural alternatives, and LayerMix multi-layer aggregation matching oracle-layer performance without oracle access.

The work proposes using the variance of specialists' task vectors across specialists as a measure of interference, yielding a pre-merge score that predicts collapse, and designs PRISM: an operator that averages task vectors first and then soft-thresholds each layer at a level set by that layer's interference; across twenty-two merge configurations from four model families only destructive merges exceed the score threshold, twelve of fourteen pre-evaluation predictions were correct, and PRISM keeps all five destructive merges above the threshold within evaluation noise of the base model without data or tuning, where plain averaging falls at least 14.4 points below it or collapses entirely.

The work proposes using the variance of specialists' task vectors across specialists as a measure of interference, yielding a pre-merge score that predicts collapse, and designs PRISM: an operator that averages task vectors first and then soft-thresholds each layer at a level set by that layer's interference; across twenty-two merge configurations from four model families only destructive merges exceed the score threshold, twelve of fourteen pre-evaluation predictions were correct, and PRISM keeps all five destructive merges above the threshold within evaluation noise of the base model without data or tuning, where plain averaging falls at least 14.4 points below it or collapses entirely.