# Synergy Head Information Drift

The work proposes HEAL, which applies causal noise intervention to multi-head outputs and counterfactual Difference-in-Differences to categorize attention heads into redundant, visual, language, and synergy types, finds that hallucinations occur when the visual-language information distribution inside synergy heads drifts away from a healthy equilibrium rather than correlating strongly with the quantity or strength of modality-specific heads, and accordingly injects dynamic calibration factors into the value vectors of synergy heads at inference time, reducing hallucinations across multiple MLLMs and benchmarks.

The work proposes HEAL, which applies causal noise intervention to multi-head outputs and counterfactual Difference-in-Differences to categorize attention heads into redundant, visual, language, and synergy types, finds that hallucinations occur when the visual-language information distribution inside synergy heads drifts away from a healthy equilibrium rather than correlating strongly with the quantity or strength of modality-specific heads, and accordingly injects dynamic calibration factors into the value vectors of synergy heads at inference time, reducing hallucinations across multiple MLLMs and benchmarks.