A heart-failure metabolite model breaks on a second NMR platform: 108 duplicate samples lift balanced accuracy from 0.64 to 0.72
Synopsis
Metabolomics models trained on one cohort often lose accuracy on another because platform, sample handling and quantification differences shift the data. Using 12,000 plasma NMR samples from the Estonian Biobank to classify heart failure (ICD-10 I50.9), a study evaluated 295 samples re-measured on a different NMR platform and calibrated alignment with up to 108 biospecimens duplicated across both. Harmonization generally beat unaligned data and stabilized as matched pairs grew, yet simple within-domain normalization stayed a strong baseline, reaching balanced accuracy 0.72 and MCC 0.46 with a support vector machine at all 108 pairs.
Interpretation
A metabolomics prediction model loses performance when it moves to a different NMR instrument and a different quantification workflow, so destination samples must be mapped back into the feature space where the model was trained. Earlier practice trained and validated within a single dataset, without an independently measured external test set, so differences from platform, operator, quantification protocol and storage were hard to assess. Training used 12,000 plasma samples measured at Nightingale Health from the Estonian Biobank; testing used 295 samples re-measured on a 600 MHz spectrometer at the Istituto Italiano di Tecnologia, including 108 biospecimens duplicated across both platforms.
Aligning domains with matched samples generally improved cross-cohort prediction over unaligned data, and performance became more stable as more matched pairs entered calibration. Without matched samples the best test result was a support vector machine on normalized data at balanced accuracy 0.64 and MCC 0.31; with all 108 matched pairs it rose to balanced accuracy 0.72 and MCC 0.46. Matched-pair counts from 0 to 108 were subsampled over 30 random trials, classifiers were trained only on the source dataset and evaluated on transformed destination data, scored by balanced accuracy and MCC.
In the pipeline, destination samples are first median-imputed and min-max scaled, then mapped toward the source domain by optimal transport, bridge mapping or empirical Bayes correction. Optimal transport learns a linear map from matched destination to matched source samples, bridge mapping fits an ordinary least squares regression predicting source features from destination features, and empirical Bayes treats the two domains as two batches and corrects feature-wise location and scale. In the matched-sample agreement assessment, optimal transport reached a Pearson correlation of 0.89, ICC 0.89 and concordance correlation coefficient 0.89, the highest among the compared methods.
Perspective
The findings rest on Estonian Biobank plasma NMR data: training samples were collected between 2002 and 2011 and measured at Nightingale Health in Finland, while the 295 test samples were collected in 2018-2019 and measured on a 600 MHz spectrometer at the Istituto Italiano di Tecnologia in Genoa. The target condition is ICD-10 I50.9 unspecified heart failure, with only 45 cases in the training set. External validation used human gestational diabetes and mouse diabetes LC-MS datasets from Metabolomics Workbench, neither containing matched samples.
Alignment is unstable when matched pairs are few: at k=1 the cross-domain transform is not sufficiently determined, and bridge mapping performs poorly at very low k. Performance depends not only on how many pairs are used but on which subset is drawn, so gains are not monotonic. Reducing distributional difference does not guarantee better classification, since global distribution matching may weaken feature-label relationships. The external datasets contain no matched samples, so matched-sample methods could not be tested for generalization there.