# LoRA Visual Experts

Hallucinations in large vision-language models partly trace to CLIP-style encoders that favor main objects and skip image detail. PLANE attaches three LoRA visual experts to CLIP, each trained on object, attribute or relation captions, and an instruction-aware gate fuses the four image representations at token level. Trained once on LLaVA-1.5 7B, it plugs into other models sharing the same encoder; on LLaVA-1.5 7B the MME hallucination subset total rises from 555.38 to 582.34 and the GIFT hallucination rate falls from 23.39% to 20.30%.

Hallucinations in large vision-language models partly trace to CLIP-style encoders that favor main objects and skip image detail. PLANE attaches three LoRA visual experts to CLIP, each trained on object, attribute or relation captions, and an instruction-aware gate fuses the four image representations at token level. Trained once on LLaVA-1.5 7B, it plugs into other models sharing the same encoder; on LLaVA-1.5 7B the MME hallucination subset total rises from 555.38 to 582.34 and the GIFT hallucination rate falls from 23.39% to 20.30%.