A radiology decision model that never writes text: open weights lift report-error detection from 72.1% to 94.7%

Radiology workflows contain many decisions that are a choice among a fixed set of answers, such as whether a report describes a finding, which diagnosis is most likely, or which subspecialty should read a study. Such decisions can now be made by a model that returns a probability for each permitted answer without generating text, with open weights that can run inside a hospital. No decision model had been specialized for radiology, and none had been compared with general-purpose decision models and large language models on the same questions; the hosted decision models Jev and OpenAI Decisions are reachable only through an external interface, so submitted data must leave the institution. On 14,142 held-out questions, RadKev-27B reached 80.8% task-averaged accuracy against 75.9% for its starting point Kev-27B, a difference of 4.9 percentage points (95% CI 3.8 to 6.2), and 3.6 points (2.4 to 5.0) over the fourteen human-labeled tasks.

The approach continues training an existing general-purpose decision model so that it keeps the same interface and differs only in its weights. The model reads a state and its questions once and returns a probability distribution over each question's options, rather than generating an answer token by token. Training used 36,109 records from public radiology sources covering report reading, case diagnosis and classification, and radiological knowledge; questions on imaging orders, triage and follow-up have no public human labels and were labeled by two models, MedGemma-27B-text and Qwen3.8-27B, and kept only when both ranked the same option first. Training ran on four NVIDIA RTX A6000 GPUs, taking 13.1 hours for RadKev-27B and 3.4 hours for RadKev-9B; on 60 benchmark records timed one request at a time, RadKev-27B answered in a median of 128 ms per question, against 16.9 s for the same backbone Qwen3.8-27B with reasoning.

The gains are concentrated in a few tasks and, for case diagnosis, partly reflect recognition of the answer options rather than reading of the case. In a control that replaced the case text with a sentence stating that no case information is available, RadKev-27B still answered 76.9% of Eurorad diagnosis questions correctly against 47.7% for Kev-27B, with 24% expected by chance; removing the imaging findings lowered subspecialty accuracy by 11.9 to 17.8 points for every system. On radiologist-annotated reports from another institution (RadGraph-XL), RadKev-27B reached 90.2% on the 2,991 definite-finding questions, 1.1 points below Kev-27B (−1.6 to −0.6), and 39.1% on the 1,046 hedged findings.

The work evaluates categorical radiology decisions on text input, covering report reading, case diagnosis and classification, and radiological knowledge across fifteen tasks and 14,142 held-out questions, with an external test on RadGraph-XL reports from another institution. The model targets decisions that must run inside an institution and whose admissible answers can be listed in advance, such as examination selection, prioritization, subspecialty routing and detection of report errors; the 27B model runs on two 48 GB GPUs and the 9B model on one. The authors propose a reader study against radiologists on institutional reports, answer spaces taken from the institution's own catalogs, and testing of error-detection and appropriateness decisions on reports and orders from other institutions.

The evaluation used public data: the test splits of Eurorad, RadCases, ReXErr, MedMCQA and MedQA derive from the same collections as their training splits, the errors in ReXErr reports were introduced by an LLM, and generalization to another institution was assessed on one external test set. Several tasks contain few questions, RSNA-RadioQA 79 and some knowledge tasks fewer after the radiology filter, so their individual estimates are less precise. The LLM comparator Qwen3.8-27B was evaluated zero-shot with a single prompt and was not fine-tuned on the same data, so the contributions of the decision format and of domain fine-tuning are not separated. Each model was trained once with a single random seed. Public examination questions and Eurorad cases may have been part of the pretraining data, but Qwen3.8-27B, Kev-27B and RadKev-27B share the same backbone, so such exposure is common to the principal comparisons. Latency was measured with one request at a time, and batched serving may change the throughput of every system.