# Multimodal Large Language Models

Multi-view 3D vision-language models suffer viewpoint-induced semantic drift: the same object and query produce weaker, diffuse or shifted target responses when the camera moves. LieVLM has geometry tokens predict local rotation parameters and channel-wise gates, applies length-preserving rotations to projected visual features within learned three-dimensional subspaces, and gates the identity-relative correction back into the original representation. On ScanRefer, LieVLM-8B reaches 64.5 Acc@0.25 and 57.2 Acc@0.5, 6.9 and 6.3 points above the published VG LLM-8B under the same proposal-refinement protocol.

Multi-view 3D vision-language models suffer viewpoint-induced semantic drift: the same object and query produce weaker, diffuse or shifted target responses when the camera moves. LieVLM has geometry tokens predict local rotation parameters and channel-wise gates, applies length-preserving rotations to projected visual features within learned three-dimensional subspaces, and gates the identity-relative correction back into the original representation. On ScanRefer, LieVLM-8B reaches 64.5 Acc@0.25 and 57.2 Acc@0.5, 6.9 and 6.3 points above the published VG LLM-8B under the same proposal-refinement protocol.

Similarity between multimodal large language model representations and human visual cortex shows that the two are alike but not which semantics they share. SAIL recasts voxel interpretation as a correspondence between each voxel and multiple sparse semantic units: it trains a unified cross-layer sparse autoencoder on several model layers, matches each voxel to several units, and compares the semantic similarity of their preferred image sets with optimal transport. Across eight subjects in the Natural Scenes Dataset, joint cross-layer training reduced unit response redundancy and raised semantic similarity over layer-wise training, in both whole cortex and selective regions of interest.

Similarity between multimodal large language model representations and human visual cortex shows that the two are alike but not which semantics they share. SAIL recasts voxel interpretation as a correspondence between each voxel and multiple sparse semantic units: it trains a unified cross-layer sparse autoencoder on several model layers, matches each voxel to several units, and compares the semantic similarity of their preferred image sets with optimal transport. Across eight subjects in the Natural Scenes Dataset, joint cross-layer training reduced unit response redundancy and raised semantic similarity over layer-wise training, in both whole cortex and selective regions of interest.

Applying the same visual-residual edit at every decoder layer with matched RMS strength reveals a persistent drop in effect, called the Evidence Anchoring Boundary; routing interventions by question type and calibration improves full-set attribute accuracy across three checkpoints, while universal subtraction is neutral or harmful.

Applying the same visual-residual edit at every decoder layer with matched RMS strength reveals a persistent drop in effect, called the Evidence Anchoring Boundary; routing interventions by question type and calibration improves full-set attribute accuracy across three checkpoints, while universal subtraction is neutral or harmful.

Multimodal large language models associate visual cues with category concepts too weakly during fine-grained recognition; LIVE tightens the candidate category list and adds category contrast hints at inference time, raising Qwen2.5-VL-7B average accuracy by 14.96 points across 13 benchmarks.

Multimodal large language models associate visual cues with category concepts too weakly during fine-grained recognition; LIVE tightens the candidate category list and adds category contrast hints at inference time, raising Qwen2.5-VL-7B average accuracy by 14.96 points across 13 benchmarks.

TiCO replaces coordinate regression in thinking-with-images with a probe token that samples evidence over visual tokens, reaching 78.0% overall accuracy on HR-Bench 4K while generating only 6.2 tokens per query.

TiCO replaces coordinate regression in thinking-with-images with a probe token that samples evidence over visual tokens, reaching 78.0% overall accuracy on HR-Bench 4K while generating only 6.2 tokens per query.

Visual token selection in multimodal LLMs now propagates scores over a visual graph and corrects for neighborhood degree: across four backbones, ten benchmarks and seven pruning ratios, LLaVA-NeXT retains 98.7% accuracy while keeping 30% of visual tokens, with a net prefill speedup.

Visual token selection in multimodal LLMs now propagates scores over a visual graph and corrects for neighborhood degree: across four backbones, ten benchmarks and seven pruning ratios, LLaVA-NeXT retains 98.7% accuracy while keeping 30% of visual tokens, with a net prefill speedup.

The authors introduce SpatialOPSD, an on-policy self-distillation framework that treats verified execution traces from a spatial coding agent as privileged information, letting a frozen teacher initialized from the same checkpoint see the traces while the student sees only raw images, with Repetition-Aware Distillation (repetition masking plus unlikelihood regularization) suppressing verbatim copying; on MindCube-Tiny and ViewSpatial, the 9B student raises average accuracy from 45.9 to 51.9, above SFT at 47.6 and GRPO at 48.2, and retains about 91% of its tool-using teacher SpatialClaw at 57.3 in a single tool-free forward pass.

The authors introduce SpatialOPSD, an on-policy self-distillation framework that treats verified execution traces from a spatial coding agent as privileged information, letting a frozen teacher initialized from the same checkpoint see the traces while the student sees only raw images, with Repetition-Aware Distillation (repetition masking plus unlikelihood regularization) suppressing verbatim copying; on MindCube-Tiny and ViewSpatial, the 9B student raises average accuracy from 45.9 to 51.9, above SFT at 47.6 and GRPO at 48.2, and retains about 91% of its tool-using teacher SpatialClaw at 57.3 in a single tool-free forward pass.

FlashGaze is a training-free method that reduces spatiotemporal redundancy before ViT encoding by using pixel-space differences as a proxy for information loss and Quadtree Dynamic Programming to jointly decide patch dropping, merging, and keeping under a fixed budget; on Qwen3-VL-8B and NVILA-8B-Video it achieves higher accuracy and lower TTFT across multiple video understanding benchmarks, and at a 20% retained ratio preserves 98% of the full-input LongVideoBench accuracy.

FlashGaze is a training-free method that reduces spatiotemporal redundancy before ViT encoding by using pixel-space differences as a proxy for information loss and Quadtree Dynamic Programming to jointly decide patch dropping, merging, and keeping under a fixed budget; on Qwen3-VL-8B and NVILA-8B-Video it achieves higher accuracy and lower TTFT across multiple video understanding benchmarks, and at a 20% retained ratio preserves 98% of the full-input LongVideoBench accuracy.

The work proposes Slow-Fast Multi-Teacher On-Policy Distillation (SF-MOPD), which couples a fast model updated directly by each teacher with a slow model that is an exponential moving average of the student, computes each teacher-induced update in log-probability space, and removes only the component pushing the fast model away from the slow model while retaining aligned and orthogonal components; experiments across multiple model scales show it mitigates capability interference, enhances specialized multimodal capabilities, and reduces average degradation on general-capability benchmarks, consistently outperforming vanilla MOPD.

The work proposes Slow-Fast Multi-Teacher On-Policy Distillation (SF-MOPD), which couples a fast model updated directly by each teacher with a slow model that is an exponential moving average of the student, computes each teacher-induced update in log-probability space, and removes only the component pushing the fast model away from the slow model while retaining aligned and orthogonal components; experiments across multiple model scales show it mitigates capability interference, enhances specialized multimodal capabilities, and reduces average degradation on general-capability benchmarks, consistently outperforming vanilla MOPD.