Shamanthak Hegde

Using activation patching across twenty-five models spanning eight LLM families, the work identifies an early-layer (L0) attention routing circuit shared by VQ-tokenized vision-language models, proposes a three-gate diagnostic that isolates ten models carrying it and rejects the other fifteen, shows via a single-variable architectural swap (LLaVA-1.6 CLIP+MLP to VQ+Linear) that vector quantization is the source of the pathological signal, and finds that only L0 ablation reduces object hallucination in open-ended generation (CHAIR_i down 31% relative) while tuned DoLA and VCD do not.

Using activation patching across twenty-five models spanning eight LLM families, the work identifies an early-layer (L0) attention routing circuit shared by VQ-tokenized vision-language models, proposes a three-gate diagnostic that isolates ten models carrying it and rejects the other fifteen, shows via a single-variable architectural swap (LLaVA-1.6 CLIP+MLP to VQ+Linear) that vector quantization is the source of the pathological signal, and finds that only L0 ablation reduces object hallucination in open-ended generation (CHAIR_i down 31% relative) while tuned DoLA and VCD do not.