A neuroimaging model that talks back: Neuro-VL beats GPT-5.4 on 16 of 17 task-group combinations and holds its ground under misleading prompts

Machine reading of neuroimaging moves from single-point classification to interactive interpretation of a whole examination: the model localizes abnormalities, describes imaging features, gives a diagnosis and generates a full report, and can discuss the same images with a user. Existing medical models often address restricted tasks, while general-purpose models lack access to private clinical imaging; the team's earlier health-system model had shown an advantage over internet-scale frontier models only in triaging examinations. On NeuroBench, spanning internal MRI and CT, MR-RATE, CTRG, 20 public brain MRI datasets, and CQ500 and RSNA, Neuro-VL beat GPT-5.4 and Gemma 4 on 16 of 17 task-group combinations and NeuroVFM on all 17, leading by 15.9, 16.9 and 23.0 percentage points when averaged equally.

The model first states which scans a study contains and their imaging characteristics, then gives a reasoning trace linking findings to specific scans, and only then answers, so responses are pushed to stay anchored to the input images. In early iterations evaluators found responses sensitive to prompt phrasing and option wording, suggesting direct answer supervision might encourage linguistic shortcuts rather than visual interpretation; later they found references to scan types that were not provided. Training used 740,000 visual instruction examples totaling 2.3 billion tokens, with questions and responses generated from radiologist reports by GPT-5 models and iteratively refined through expert feedback; adding the imaging statement and reasoning trace lowered training loss and improved out-of-distribution performance.

A longitudinal version reads the current and prior studies together to judge changes in findings, disease evolution, and interventions between studies and their imaging changes. Single-study interpretation cannot distinguish new findings from longstanding abnormalities, while comparing with prior imaging is a routine part of radiological reading. On a longitudinal benchmark of 734 current-prior study pairs and 2,294 questions and report tasks, Neuro-VL-Longitudinal reached 87.5% and 96.9% accuracy on change characterization and disease trajectory assessment, above the best corresponding GPT-5.4 results of 81.6% and 86.5%; intervention characterization reached 80.3%, above the best Gemma 4 result of 68.9%.

The model is trained to correct unsupported assumptions in user prompts, so it answers from the images even when a prompt implies pathology or supplies a wrong diagnosis or location. In high-stakes clinical settings, interactive systems may place undue weight on user-supplied presumed diagnoses or suggestive histories at the expense of objective imaging findings. On 320 studies reported as normal, Neuro-VL succeeded 95.6% of the time when prompts assumed pathology, versus 64.1% for GPT-5.4; on 246 expert-annotated brain MRI studies, replacing the correct diagnosis with an incorrect one cut Neuro-VL success from 95.1% to 90.2% and GPT-5.4 from 99.6% to 44.5%.

The demonstrated scope is training on neuroimaging acquired during routine clinical care at Michigan Medicine between 1995 and 2025, and retrospective evaluation on internal held-out studies and 24 international public datasets covering localization, feature description, diagnosis, report generation, longitudinal comparison and robustness to misleading prompts; the authors state that a clinician-Neuro-VL interaction study is underway and that prospective evaluation is planned to establish real-world safety, generalizability and clinical utility.

The authors list several limitations: Neuro-VL currently lacks agentic capabilities such as independent retrieval of additional patient data, guideline consultation, tool use and iterative verification; it was developed at Michigan Medicine, and although evaluation covers international public datasets, the evaluations are retrospective and cannot fully characterize performance under changing clinical populations, acquisition protocols or workflows; and generated reasoning traces and attention visualizations make outputs easier to inspect but do not establish that the reported rationale faithfully reflects the model's underlying computations, so better mechanistic interpretability is needed to link responses to input tokens.