Visual edits lose their grip mid-decoder: equal-strength interventions fade before answers settle, and calibrated routing turns that into attribute accuracy

The effect of a visual intervention changes with decoder depth and enters a persistent low-effect region whose onset is named the Evidence Anchoring Boundary (EAB), the layer where a given edit's influence on the final answer starts to fade. Earlier work locates promising intervention sites from attention maps or intermediate states, but a readable layer is not the same as a layer where a specified edit still changes the final answer. Across six checkpoints from three model families, the same visual-residual edit is applied at successive layers with root-mean-square strength matched across depth; with the boundary frozen on calibration queries, held-out effects on disjoint evaluation queries before EAB are larger than those after it, and every image-cluster confidence interval excludes one.

Answer-preference stabilization and visual-edit effectiveness are separate processes: the model keeps developing its answer preference after the tested edit has largely lost influence. A natural explanation would be that the effect drops only because the answer is already settled, but intermediate readouts show otherwise. Comparing EAB with the Answer Commitment Point (ACP), the layer after which decoded margin signs and candidate rankings stay aligned with the final prediction, readouts can keep improving after EAB; in a controlled audit of 96 triplets per model, the windowed edit corrected two LLaVA-7B baseline errors and zero for the other three checkpoints.

Whether counterfactual subtraction helps depends on which preference it removes, so the same operation can be corrective, redundant, or destructive across question types. Subtraction had been treated as a broadly applicable correction, yet universal subtraction is neutral or harmful on the full attribute sets. On the AMBER hard-attribute subset whose counterfactual favors the incorrect candidate, subtraction raises pair accuracy on LLaVA-7B, LLaVA-13B, and InternVL; on the full attribute sets the changes are neutral or negative, and direct-contact relations decline on the two LLaVA checkpoints.

Calibrated selective routing preserves the original prediction by default and applies an intervention only when calibration supports it, concentrating the benefit of subtraction where it is usable. Universal subtraction changes full-set attribute accuracy by neutral or harmful amounts on three checkpoints, while routing improves it on the same images. Routing improves full-set attribute accuracy on LLaVA-7B, LLaVA-13B, and InternVL, with all three image-cluster confidence intervals above zero; the rule is selected by five-fold image-disjoint cross-validation, so each image is scored by a rule that never saw it.

The report is aimed at researchers and engineers working on hallucination mitigation in multimodal large language models, and covers four base checkpoints, LLaVA-1.5-7B/13B, Qwen2.5-VL-7B, and InternVL3.5-8B, with the depth trend replicated on LLaVA-NeXT-34B and Qwen2.5-VL-32B. EAB describes the effect trajectory of the tested visual-residual edit at a given checkpoint; calibration and evaluation use disjoint images, and the boundary is frozen before held-out evaluation. Routing gains are established mainly for binary attribute claims, and the authors propose measuring the chosen operator's influence at the intended depth and evaluating the resulting decisions on the target task before applying a visual correction.

The boundary location comes from one operational rule, and the 48/72 calibration-to-evaluation split supports the reported effect contrast, but the authors state that a larger, more diverse sample is needed to estimate how precisely the boundary generalizes across settings. The numerical results cover one boundary rule and one primary Q1 operator, and new models, prompts, operators, or task categories require recalibration. PEC is used only in its calibrated POPE-adversarial setting, and the frozen configuration loses accuracy when transferred to TextVQA and MME. Additional counterfactual passes add compute cost, and wall-clock latency and FLOP overhead remain unmeasured.