# Video-Language Models

On CARLA third-person driving videos, six open-source video-language models judging forward/backward, left/right, and accelerate/decelerate stayed near chance under standard, visual, few-shot, and chain-of-thought prompting, with systematic class biases such as every model scoring 100% on forward and 0% on backward. LoRA finetuning then reached near-perfect held-out accuracy for all models except VideoLLaMA2 on accelerate/decelerate, and simple classifiers on pre- and post-compression visual features decoded motion cues at high accuracy. The motion information is present inside the models, but the prompted prediction pathway does not reliably use it.

On CARLA third-person driving videos, six open-source video-language models judging forward/backward, left/right, and accelerate/decelerate stayed near chance under standard, visual, few-shot, and chain-of-thought prompting, with systematic class biases such as every model scoring 100% on forward and 0% on backward. LoRA finetuning then reached near-perfect held-out accuracy for all models except VideoLLaMA2 on accelerate/decelerate, and simple classifiers on pre- and post-compression visual features decoded motion cues at high accuracy. The motion information is present inside the models, but the prompted prediction pathway does not reliably use it.

The work presents AbSteering, a two-stage framework combining abnormality-centric chain-of-thought training with a Direct Preference Optimization objective for fine-grained abnormality discrimination, to adapt general-purpose VideoLMs to high-resolution CT report generation, and curates the CT-RATE-AB dataset; results show that general-purpose VideoLMs transfer effectively to 3D medical imaging under limited data, achieving state-of-the-art performance on fine-grained clinical efficacy metrics, with superior detection sensitivity over domain-specific CT foundation models pretrained on large-scale CTs while mitigating hallucinations.

The work presents AbSteering, a two-stage framework combining abnormality-centric chain-of-thought training with a Direct Preference Optimization objective for fine-grained abnormality discrimination, to adapt general-purpose VideoLMs to high-resolution CT report generation, and curates the CT-RATE-AB dataset; results show that general-purpose VideoLMs transfer effectively to 3D medical imaging under limited data, achieving state-of-the-art performance on fine-grained clinical efficacy metrics, with superior detection sensitivity over domain-specific CT foundation models pretrained on large-scale CTs while mitigating hallucinations.