PAC-Private Diffusion Adaptation Perturbs the Learned Component Once, Beating DP-SGD on Subject Identity and Generation Quality at Equal Reconstruction Privacy

It proposes a diffusion model adaptation framework that targets reconstruction privacy (RP) directly through PAC-privacy, instead of reaching RP indirectly via DP. Existing DP approaches such as DP-SGD bound output sensitivity to changes in individual records, which relates to RP only indirectly, whereas PAC-privacy is defined directly with respect to posterior advantage over the prior and therefore implicates RP directly. The argument is made at the level of definitions, with the indirection between DP and RP identified as an important source of utility loss; no formal theorem or proof detail is given.

It provides a scalable high-dimensional PAC privatization: learn a compact data-dependent component, then calibrate anisotropic Gaussian noise from the covariance of repeated mechanism outputs. Unlike DP-SGD, which repeatedly clips gradients and injects noise across updates, the method perturbs the learned component only once after optimization, avoiding privacy composition across gradient updates. The method description specifies LoRA or Textual Inversion for the learned component, covariance-based noise calibration, and a single perturbation; no algorithmic complexity or hyperparameter details are reported.

On few-shot concept personalization and full-dataset image synthesis, the method better preserves subject identity, generation quality, and downstream classification accuracy than DP while achieving the same reconstruction privacy. Relative to the DP baseline, the improvement spans three utility dimensions at once rather than trading privacy for utility. Evaluation covers two task settings and reports a comparison against DP; the summary provides no numeric metric values, dataset names, or sample sizes.

The result targets settings where diffusion models are trained or adapted on sensitive data and synthetic images are to be released, such as few-shot concept personalization and full-dataset image synthesis; the method presupposes learning a compact component via LoRA or Textual Inversion and applies to diffusion models coverable by such adaptation. For practitioners seeking to replace stepwise DP-SGD noise with a single perturbation and thereby avoid privacy composition, this route offers an actionable alternative.

The summary reports no numeric evaluation metrics, dataset names, sample sizes, or privacy parameter settings, so the basis for the claim of equal reconstruction privacy and the size of the utility gain still need to be checked in the full text. Whether PAC-privacy noise calibration remains stable in high dimensions, how the single perturbation behaves across different adaptation components, and how robust the method is to stronger reconstruction attacks are open questions a reader may follow. In addition, this assessment is based on the summary only; figures and experimental details were not included.