Radiology agents limited to cleared tools recover 29% of actionable findings when planning from the indication alone
Synopsis
A radiology agentic framework separates a large-language-model orchestrator from a modular tool interface and uses 1,989 finding classes consolidated from FDA-cleared radiology AI devices as its tool catalog, simulating structured report generation on the PARROT fictional-report corpus. Planning from the clinical indication alone, actionable-finding recall is 0.290 with the cleared catalog and 0.437 with an unlimited catalog; when the planner reads the target report, these rise to 0.426 and 0.740. Tool outputs are simulated from the reference report, so the comparison isolates tool coverage and orchestration rather than tool error.
Interpretation
Radiology report generation is measured in two layers: whether the tool catalog contains the relevant finding class, and whether the orchestrator calls tools it already has. Previously published radiology agentic systems were tied to one modality or one anatomy with a fixed tool set; here the same orchestrator switches between a cleared catalog and an unlimited catalog. On 1,059 PARROT reports with parseable outputs, giving only the clinical indication yields actionable-finding recall of 0.290 with the cleared catalog and 0.437 with the unlimited catalog.
The orchestrator is a staged pipeline rather than a single LLM call: context loading, planning, grouping, per-track execution, merging, drafting and review follow one another. The planner turns the clinical indication and a retrieved RadReport template into an ordered list of analysis steps, each carrying a coverage manifest naming the anatomical regions and observation categories the report must cover. The grouper places steps on the same lesion or series into one sequential track and runs independent series concurrently; each track runs a ReAct loop, in which the model alternates between reasoning and calling a tool, each result feeding the next decision.
Missed findings are attributed one by one, and most are catalog gaps rather than orchestration failures. In the cleared-catalog mode with only the clinical indication, 61.8% of the 9,675 missed or incorrectly reported findings have no cleared class covering them on the study's modality, and 31.9% had a tool offered but not called. By modality, the catalog and orchestration shares are 59% and 35% for CT, 68% and 27% for MR, 54% and 38% for radiography, and 64% and 24% for ultrasound.
Perspective
Evaluation runs on 1,059 English fictional PARROT reports spanning CT, MRI, radiography and ultrasound, and every tool call is simulated from the reference report, so the conclusions concern tool coverage and orchestration rather than real detector performance.
The authors note that PARROT reports are fictional and not a proxy for real clinical situations; the held-out set has only 100 reports, enough for the direction of the effect but not to rank the cutoffs; the catalog is an FDA snapshot through March 2026, the main orchestrator is a single model, and absolute numbers will move with the model.