DP-RGMI splits differential privacy's performance loss on 594,000 chest X-rays into encoder geometry and task-head utilization
Synopsis
The authors introduce DP-RGMI, a framework that treats differentially private training as a structured transformation of representation space and decomposes performance degradation into representation displacement, spectral effective dimension, and a utilization gap defined as the difference between linear-probe and end-to-end AUROC; across more than 594,000 chest X-rays from four datasets and three pretrained initializations (ImageNet, DINOv3, MIMIC-CXR), with PadChest as the primary dataset (110,525 frontal images, 22,045 test images), they find that strong privacy consistently leaves linear separability largely preserved while a utilization gap persists (G = 8.0 for ImageNet at ε = 1.0, 3.4 for MIMIC at ε = 0.7, 6.1 for DINOv3 at ε = 0.
Fig. 1. Overview of DP-RGMI framework decomposing DP training into representation displacement ∆(ε), spectral structure deff(ε), and utilization gap G(ε).
· Page 3Interpretation
DP-RGMI decomposes performance degradation under differential privacy into encoder geometry (representation displacement Δ and spectral effective dimension d_eff) and task-head utilization (the gap G between linear-probe AUROC and end-to-end AUROC). The privacy-utility trade-off in medical imaging has been evaluated almost exclusively through end-to-end metrics such as AUROC or Dice, which cannot reveal whether privacy noise reduces linear separability, reshapes representation geometry, or mainly impairs task-head optimization; this framework separates the three and provides a reproducible diagnostic workflow (Algorithm 1). The framework is explicitly defined and model- and dataset-agnostic, requiring only embeddings and standard evaluation metrics; it is executed under one protocol on the primary PadChest dataset and two generalization datasets, with uncertainty from 1000 bootstrap resamples.
Under strong privacy, linear-probe AUROC remains consistently higher than end-to-end AUROC, yielding a stable utilization gap G, which indicates that discriminative structure persists in the private encoder but is not fully exploited during joint training. The non-private baseline gives G(∞) ≈ 0, whereas under privacy G is clearly positive (8.0 for ImageNet at ε = 1.0, 3.4 for MIMIC at ε = 0.7, 6.1 for DINOv3 at ε = 0.7), replacing a vague 'privacy collapses representations' account with a measurable distinction between preserved representation and under-utilization. Reported as mean ± standard deviation on the 22,045-image PadChest test set with 1000 bootstrap resamples; a positive G(ε) is also observed when the same protocol is repeated on CheXpert and ChestX-ray14.
Representation displacement and spectral effective dimension follow non-monotonic, initialization- and dataset-dependent trajectories under privacy rather than uniform degradation. Under ImageNet initialization d_eff decreases at moderate privacy (3.4 at ε = 8.6) but increases at stronger privacy (9.2 at ε = 1.0); DINOv3 trends toward lower effective dimension as privacy strengthens (5.1 → 3.9), while MIMIC increases gradually; displacement magnitude also does not map monotonically to utility. Supported by the reported Δ and d_eff values for the three initializations on PadChest (e.g., DINOv3 Δ = 1.9; MIMIC rising from Δ = 0.1 to about 1.3–1.4) and by the trajectory figures for CheXpert and ChestX-ray14.
The association between the utilization gap and end-to-end performance is robust across datasets but varies by initialization, while geometric quantities capture additional prior- and dataset-conditioned variation. Across datasets AUROC is negatively associated with G (PadChest ρ = −0.95, CheXpert ρ = −0.86, ChestX-ray14 ρ = −0.98), but across initializations it is ρ = −0.78 for ImageNet, ρ = −0.31 for MIMIC, and ρ = +0.55 for DINOv3; under MIMIC initialization Δ correlates with AUROC at ρ = +0.81. Spearman rank correlations based on n = 9 privacy-budget × dataset or initialization combinations per setting and n = 27 overall; the authors state that the association with G partly reflects its definition relative to AUROC and interpret it descriptively rather than causally.
Perspective
The framework targets medical imaging researchers and deployers who must choose privacy budgets in settings involving cross-institutional reuse, transfer learning, or frozen feature extraction, and it applies to settings such as multi-label chest X-ray classification where embeddings and standard evaluation metrics are available; it is model- and dataset-agnostic by construction, and the authors expect similar geometry-utilization interactions wherever representations are reused or fine-tuned. In practice, if two privacy budgets yield similar end-to-end AUROC but one shows a larger G, the framework indicates that recoverable signal persists, pointing to optimization-side interventions such as freezing the encoder, retraining only the head, or adjusting clipping for head parameters, rather than relaxing privacy; if Δ is large while probe performance remains stable, the representation has moved substantially from its pretrained prior, which may affect transfer or reuse across institutions; if d_eff drops markedly, spectral concentration rises and representational diversity falls, potentially limiting adaptation to new tasks, in which case revisiting pretraining or privacy strength may be more appropriate.
The authors note that the association between G and AUROC partly reflects G's own definition, so it is interpreted descriptively rather than causally; the correlation analysis uses only n = 9 per setting and n = 27 overall, and the sign even reverses across initializations (ImageNet ρ = −0.78 versus DINOv3 ρ = +0.55), suggesting the relationship depends on the pretrained prior. Privacy guarantees are applied at the image level rather than the patient level because training samples correspond to individual radiographs, and how this granularity affects clinical interpretation deserves attention. All experiments use a single ConvNeXt-Small backbone, which the authors describe as a constrained rather than free choice: batch normalization is incompatible with the per-sample gradients of DP-SGD, and transformers show unstable DP optimization; the framework's behavior in other tasks such as segmentation remains to be validated. In addition, the privacy cost of interventions such as head-only re-optimization has not been quantified and is listed by the authors as direct future work.
