A planner-executor agent now answers lung screening CT questions, deferring 92.6% of out-of-scope ones to doctors

Questions patients raise after receiving a lung cancer screening CT report can be answered by a multimodal agentic system that invokes imaging tools and hands some questions to a physician. Earlier general-purpose model approaches could only rewrite reports in patient-friendly language or answer general screening questions, and could not integrate specialized imaging tools; existing medical agents mostly targeted clinical decision-making or diagnosis rather than patient communication. On 116 lung cancer screening CT reports and images from a tertiary academic hospital, overall tool-calling accuracy was 93.1% (646 of 694; 95% CI 91.2%–95.0%) versus 78.5% (545 of 694; 75.5%–81.4%) for the single-agent baseline, with all multi-agent variations significantly higher (all P < .001).

The system splits a patient question into a planning step and an execution step, where the planner decides which tool to call or whether to defer, and the executor retrieves images, consults guidelines, and writes the answer. Separating planning from execution assigns logical judgment and content generation to different agents, leaving room for a dedicated verification step. Both agents run on a HIPAA-compliant GPT-4o; the executor can call a Python slice-retrieval function and the open-source segmentation model TotalSegmentator, and refers to Lung-RADS v2022 guidelines; few-shot prompting for both agents performed best, and planner prompting strategy affected accuracy more than executor prompting.

The system actively defers questions that need patient-specific history, prognosis, or more information instead of answering them itself. The single-agent baseline reached 64.7% (323 of 499; 60.5%–68.9%) on the same deferral judgment, and the multi-agent architecture was significantly higher. Across 499 text-based responses judged by an LLM-as-a-judge, overall self-regulation accuracy was 92.6% (462 of 499; 90.2%–94.8%), with 99.6% recall on answerable questions and 99.5% precision on defer-to-doctor questions.

Four physicians rated clinical quality, and answerable and defer-to-doctor responses came out close on clinical accuracy. Ratings covered factuality, conciseness, and patient understandability, plus comprehensiveness for answerable questions and relevance for defer-to-doctor questions. Among 100 proportionately sampled responses, 78.9% (180 of 228) of answerable responses received perfect 5.0 scores for clinical accuracy from all readers and 73.2% (167 of 228) for patient understandability; defer-to-doctor responses reached 77.3% (133 of 172) for clinical accuracy, not significantly different from answerable (P = 0.715), and 47.1% (81 of 172) for conciseness, lower than answerable (P = 0.027).

The system was evaluated on 116 lung cancer screening CT reports and 699 simulated patient questions, with five samples excluded from tool-calling analysis because the reference standard was ambiguous, and physician ratings covering 100 proportionately sampled responses. The intended setting is patient communication about lung cancer screening results; the authors propose using it as a standalone tool with appropriate regulatory safeguards or alongside a radiologist or screening team clinician. Stated next steps include collecting a broader range of questions directly from patients, adding follow-up capability for multi-turn conversations, expanding to additional tools such as nodule detection algorithms, and extending the planner-executor framework to other medical imaging domains and clinical communication workflows.

Evaluation used LLM-assisted simulated patient questions rather than questions sourced directly from patients, so ambiguous queries that would need clarification in real clinical settings are not well covered. The system has no multi-turn capability and cannot ask follow-up questions about ambiguous queries. Defer-to-doctor responses scored lower on perfect conciseness than answerable ones, which the authors attribute to the system providing relevant clinical context before deferring, making those responses longer. Error analysis found tool-calling errors mostly involved inaccurate series or image numbers or segmenting unrelated anatomical structures, while self-regulation errors included absolute statements about patient-specific treatments and minor misinterpretation of Lung-RADS guidelines or related medical terms.