Echocardiography diagnosis stops reading whole videos: one token per measurement lifts average AUC from 0.659 to 0.748

A multi-view echocardiography study is now factorized into measurement-specific tokens, one per measurement, and a diagnosis reads only the tokens relevant to it. Earlier echo foundation models either trained directly on diagnosis labels or used text reports for contrastive learning, so measurements appeared only as discrete text and could not be selected by a downstream task. On MIMIC-IV-Echo, against the EchoPrime initialization shared by all baselines, average AUC rises from 0.659 to 0.748; on the internal cohort it rises from 0.730 to 0.778, improving 7 of 8 diagnoses.

Each measurement token uses cross-attention to pull information only from the views relevant to that measurement, and a prediction head reconstructs the measurement value from the token. Contrastive learning collapses the whole measurement set into a single tabular embedding aligned with a single echo embedding, so individual measurements cannot be factorized within the learned representation. Training uses a masked, variance-normalized mean squared error over observed (study, measurement) pairs; the two datasets are trained separately, with MIMIC-IV-Echo holding 4,514 patients and 7,144 studies and the internal cohort 7,404 patients and 9,431 studies.

The attention patterns over echo views largely match the measurement-view relationships cardiologists use. No earlier method represented measurements as an intermediate layer, so there was no measurement-view attention to compare against. On MIMIC, EchoTab selects the same views as the cardiologist in 16 of 20 measurement-view pairs (80%) and agrees on 13 of 16 measurements (81%); the three disagreements involve biplane LVEDV, LVEF and TAPSE, and the alternative views it favors also capture the relevant cardiac structure.

The report covers echocardiography specifically: the model is trained and evaluated separately on MIMIC-IV-Echo v1.0 and an internal cohort, and the diagnosis tasks are ordinal severity grades held out from the structured measurement labels, spanning valvular disease to systolic dysfunction. Building a diagnosis-specific embedding requires specifying in advance which measurements are relevant to each diagnosis, a mapping defined by a cardiologist. The authors propose that the representation learning framework is in principle modality-agnostic and could be tried on other multi-view imaging studies with quantitative measurements, such as cardiac MRI, CT or retinal imaging.

The authors note that the mapping between diagnoses and relevant measurements must be specified by hand, which introduces domain knowledge into the model and requires specifying relevant measurements when extending the framework to new diagnoses. Only one strategy for combining diagnosis-relevant measurement tokens was explored, namely averaging them into a diagnosis-specific study embedding; the authors suggest learned weighting or attention may better capture interactions among measurement tokens. In the attention analysis, measurements that depend on view types absent from the view set or on Doppler views were excluded, because the view classifier groups all apical Doppler acquisitions into a single category too coarse for this analysis.