Multimodal models stop guessing at fine-grained recognition: tighter candidate lists plus contrastive hints lift 13 benchmarks by about 15 points

Multimodal large language models underperform on fine-grained recognition because visual cues and category concepts are insufficiently associated at inference time, not because perception or category knowledge is missing. Most prior work relies on domain-specific post-training to add fine-grained ability, at the cost of generalization to unseen domains; LIVE leaves parameters frozen and adjusts only the candidate category list and the recognition guidance. Across 13 benchmarks against 24 compared methods, LIVE raises average accuracy by 14.96 points for Qwen2.5-VL-7B and 11.64 points for Qwen3-VL-8B.

A compact candidate list plus guidance aimed at confusable categories makes it easier for the model to bind discriminative visual cues to the correct category. Taking a fixed top-k by similarity struggles to balance covering the ground-truth category against suppressing competing categories; LIVE first ranks by visual relevance, then recalls likely ground-truth categories through inter-category confusion relations. On Flower under the 4-shot setting, LIVE keeps 7.5 candidates on average while retaining over 97% ground-truth coverage, whereas a fixed top-10 stays below 90%.

Category knowledge is organized into reusable skills, and detailed content is invoked only when recognition uncertainty calls for it. Attaching detailed guidance to every candidate introduces interference; LIVE first recognizes with category names and short descriptions, then for high-entropy predictions uses first-token matching to find confused categories and appends their visual checklists and comparative knowledge. On Flower and CUB, targeted invocation cuts the average number of categories receiving detailed guidance from 10.28 to 6.99 and raises average accuracy from 55.63% to 57.00%.

The report is for researchers and engineers working on visual recognition and multimodal reasoning, covering fine-grained image recognition, fine-grained action recognition, and general plus out-of-distribution image recognition, evaluated on 13 benchmarks with Qwen2.5-VL-7B and Qwen3-VL-8B backbones and CLIP ViT-B/16 or SigLIP supplying visual relevance. The method updates no model parameters and needs a few reference samples per category to build inter-category confusion relations and category skills, using a 4-shot reference set by default. The authors propose to examine whether similar phenomena appear in visual question answering, visual grounding and video temporal grounding, and to extend the method to broader recognition scenarios.

The phenomenon and its mitigation rest on the assumption that category-to-image attention can serve as a proxy for the association strength between visual cues and category concepts, supported by observed attention and recognition changing together. The low-risk claim for candidate reduction depends on the ground-truth retention probability meeting a theoretical threshold, and measured coverage is high in the evaluated settings but not guaranteed. Hyperparameters such as the entropy threshold and candidate budget are fixed across all datasets, and the authors report stable results over a reasonable range without guaranteeing behavior under larger distribution shifts.