Skip to main content

# Medical Visual Question Answering

2 related articles

Subscribe to updates

Nature Medicine Oct 29, 2026
A medical vision-language model that scores 0.895 on diagnosis and points back to the lesion
Medical imaging foundation models have traded generality against pixel-level precision, and most return a label without evidence. EVLF-FM feeds both disease-level and pixel-level visual features into a language model, first with supervised fine-tuning and then with reinforcement learning on verifiable rewards, spanning six specialties and 10 imaging modalities. On internal diagnostic validation it reached the highest mean accuracy among evaluated models, 0.895, and averaged 0.842 Acc@0.5 for medical visual grounding across nine modalities.
Medical imaging foundation models have traded generality against pixel-level precision, and most return a label without evidence. EVLF-FM feeds both disease-level and pixel-level visual features into a language model, first with supervised fine-tuning and then with reinforcement learning on verifiable rewards, spanning six specialties and 10 imaging modalities. On internal diagnostic validation it reached the highest mean accuracy among evaluated models, 0.895, and averaged 0.842 Acc@0.5 for medical visual grounding across nine modalities.
SingHealth Community Hospitals Stanford University Catholic University of Korea National University of Singapore Singapore National Eye Center Hokkaido University Nanyang Technological University Aier Eye Hospital Group (China)
Yang Bai（First author）Darren Shu Jeng Ting（Corresponding author）Daniel Shu Wei Ting（Corresponding author）
EVLF-FM Visual Grounding Medical Visual Question Answering GRPO Explainable AI
arXiv Sep 25, 2026
ClinCoT pushes preference optimization from answer-level correction down to lesion-region reasoning: it beats MMedPO and other baselines on most metrics across SLAKE, VQA-RAD and IU-Xray, with more consistent gains after SFT initialization
The work proposes ClinCoT, a clinical-aware visual chain-of-thought framework: it uses hypotheses-driven region proposals (disease-conditioned activation maps from a clinical-aware tool such as MedKLIP, thresholded into candidate regions), has the target Med-VLM generate multiple region-conditioned reasoning chains by jointly processing the original image and each candidate region, scores them with two Med-LLM evaluators that combine current-step score with next-step influence and penalize evaluator disagreement, and trains with a score-margin-aware DPO plus iterative regeneration of preference data; across SLAKE, VQA-RAD and IU-Xray it outperforms DPO, Self-Rewarding, STLLaVA-Med, POVID, SIMA, FiSAO and MMedPO on most metrics, is strongest on report generation, falls below MMedPO on VQA-R
The work proposes ClinCoT, a clinical-aware visual chain-of-thought framework: it uses hypotheses-driven region proposals (disease-conditioned activation maps from a clinical-aware tool such as MedKLIP, thresholded into candidate regions), has the target Med-VLM generate multiple region-conditioned reasoning chains by jointly processing the original image and each candidate region, scores them with two Med-LLM evaluators that combine current-step score with next-step influence and penalize evaluator disagreement, and trains with a score-margin-aware DPO plus iterative regeneration of preference data; across SLAKE, VQA-RAD and IU-Xray it outperforms DPO, Self-Rewarding, STLLaVA-Med, POVID, SIMA, FiSAO and MMedPO on most metrics, is strongest on report generation, falls below MMedPO on VQA-R
Mohamed bin Zayed University of Artificial Intelligence East China Normal University
Xiwei Liu（First author）Yutong Xie（Corresponding author）
ClinCoT Chain-of-Thought Preference Optimization Medical Vision-Language Models Medical Visual Question Answering

You’re all caught up