Chest X-ray reports stop being one long paragraph: align images to disease labels, then let an LLM rewrite them as Q&A

The output format changes from a free-text narrative to itemized question-answer pairs, each question targeting one clinical finding with an answer grounded in image evidence, followed by a short diagnostic summary. Prior methods generate long free-text narratives in which key findings sit inside paragraphs and are hard to retrieve; the Q&A reformulation aims at structural clarity rather than replacing narrative reports. On the two public chest X-ray benchmarks MIMIC-CXR and IU X-Ray, the framework outperforms compared methods on most lexical and clinical metrics, including multi-view method MPO and Q&A-style baseline CoD.

Image and text representations are not learned independently; predicted disease labels pull similar cases together and push dissimilar ones apart, so the shared space is organized by diagnostic similarity. Earlier cross-modal alignment relied on image-report pairs and did not use diagnostic labels as supervisory signals for joint embedding learning; this framework puts predicted diagnostic labels directly into the contrastive objective. Ablation shows replacing CLIP-style alignment with disease-aware visual-textual contrast yields stronger B4 gains, and adding the prior-guided attention module further improves lexical and clinical metrics.

Generation runs in two steps: a decoder produces an intermediate report, a medical knowledge graph retrieves clinical evidence, and a fine-tuned LLM restructures the result into Q&A pairs plus a summary. Prior knowledge-injected report frameworks add external clinical text during decoding; this framework places graph retrieval before Q&A synthesis and retrieves connected evidence rather than isolated fragments. The knowledge graph is built from roughly 5,000 thoracic-imaging PubMed abstracts, and retrieval is bounded to two hops; ablation shows removing retrieval seed initialization causes the largest CheXbert-F1 drop.

The framework targets 2D chest X-rays, with training and evaluation on the two public datasets MIMIC-CXR and IU X-Ray, and disease labels limited to the first 13 CheXbert categories. Cross-institution evaluation is zero-shot on ReXGradient-160K, spanning 3 health systems and 79 medical sites, with no training or fine-tuning on that dataset. The authors propose extending to 3D modalities such as CT and MRI, and investigating explicit temporal modeling and broader multi-center data.

The authors note that the prior-guided attention module does not explicitly encode the temporal order or time intervals between prior scans, so it captures historical context but does not directly model longitudinal progression. The large language model serves as a fixed restructuring component rather than being jointly optimized for multimodal reasoning. Training is limited to MIMIC-CXR and IU X-Ray, which may constrain the learned representations to their acquisition protocols and population characteristics. The framework relies on paired images, reports and diagnostic labels, which may limit scalability in data-scarce settings. Noise-injection experiments show performance degrading across all stages as label noise rises, so improving disease prediction remains a direction for strengthening alignment reliability.