Medical image pretraining stops reading only reports: DICOM acquisition fields enter contrastive learning, lifting zero-shot CT macro-AUROC by 1.41 points
Synopsis
Medical vision-language pretraining learns from paired images and radiology reports, but scanner and acquisition-protocol differences can impair transfer. Meta-CLIP uses structured acquisition information in DICOM headers as a training-time prior: a kernel over numerical and categorical fields defines a bounded pairwise margin that reweights cross-modal negatives during pretraining, while inference keeps the standard dual-encoder architecture and needs no metadata. On CT-RATE it improves macro-AUROC over CT-CLIP by 1.41 points, with a 95% paired bootstrap interval of 0.96 to 1.85 points, and reaches the highest macro-AUROC on four of five chest radiography benchmarks.
Interpretation
Medical vision-language pretraining now treats acquisition conditions such as scanner and protocol as a training signal, not only the clinical semantics carried by reports. Existing approaches mainly enrich clinical semantics, leaving structured acquisition information in DICOM headers underused, while scanner and protocol differences can impair transfer. On CT-RATE, Meta-CLIP improves macro-AUROC over CT-CLIP by 1.41 points, with a 95% paired bootstrap confidence interval of 0.96 to 1.85 points.
The mechanism changes negative-sample weighting during pretraining so that studies sharing imaging conditions are distinguished more finely. A kernel over numerical and categorical DICOM fields defines a bounded pairwise margin that increases the contribution of acquisition-similar negatives, emphasizing distinctions among studies that share imaging conditions. The kernel remains fixed during image-text optimization, and inference follows the standard dual-encoder architecture without requiring metadata; evaluation covers zero-shot classification, appearance perturbations, grounding and representation analyses on chest radiography and volumetric chest CT.
Acquisition metadata as a training-time prior was also probed on a second CT backbone. Beyond CT-RATE and the chest radiography benchmarks, an exploratory BrgSA evaluation extends the analysis to another CT backbone. Meta-CLIP achieves the highest macro-AUROC on four of five chest radiography benchmarks.
Perspective
The demonstrated scope is zero-shot classification, appearance perturbations, grounding and representation analyses on chest radiography and volumetric chest CT, plus an exploratory BrgSA evaluation extending to another CT backbone; inference follows the standard dual encoder and requires no metadata.
The CT-RATE gain is reported as 1.41 points with a 95% paired bootstrap interval of 0.96 to 1.85 points, chest radiography is highest on four of five benchmarks, the BrgSA evaluation is explicitly exploratory, and code will be made publicly available upon publication.