Noise-only distillation is no sanitizer: a shared public checkpoint lets a student inherit its teacher's memorized labels and backdoors
Synopsis
Distilling a teacher on pure Gaussian noise still hands the student the teacher's memorized training records and planted backdoors whenever both start from the same public initialization. On MNIST, distilling only never-trained auxiliary outputs yields a membership-inference AUC of 0.86 and a 93% backdoor attack-success lift, while an independently initialized student stays near chance. In a fine-tuning threat model, an attacker with the public base but no data recovers the memorized labels of 93% of private canaries from noise queries alone.
Interpretation
A student queried only on noise statistically independent of the private data still inherits the teacher's membership signal on individual records, its memorized wrong labels and its poisoning backdoor, so data-free distillation is not a privacy or security boundary. Data-free distillation had been treated as a defense that removes memorized records and backdoors, on the grounds that the student never sees a private record and the queries carry no private information. On MNIST with an MLP, the all-logit noise student reaches membership AUC 0.988 against the teacher's 0.989, reproduces the memorized wrong label for 65.7% of member canaries versus 1.0% of non-members, and matches the teacher's 99.4% backdoor attack-success lift.
Releasing only auxiliary outputs that were never trained and carry no class information still leaks under a shared initialization, while the identical channel carries nothing from an independent one. A function-space intuition says semantically empty outputs should carry no information about private data. On MNIST the auxiliary noise channel gives membership AUC 0.862 and a 93.0% backdoor lift, versus 0.535 and 13.9% from an independent initialization; distilling the auxiliary channel of a clean teacher trained from the same initialization gives AUC 0.454 and zero backdoor lift, locating the signal in the private data.
When a provider fine-tunes a public base on private data, an attacker who owns the base and holds no domain data can copy the private fine-tune's memorized labels and backdoor into a student using noise queries. Earlier model-extraction work aimed at reproducing functionality, whereas what is copied here is the privacy-relevant idiosyncrasy of the private fine-tuning set. On MNIST the shared-base attack recovers the memorized label of 92.7% of member canaries (private model 93.7%, non-members 0.7%) at calibrated membership AUC 0.983, while an equally accurate sibling base trained with a different seed recovers 8.3%; on CIFAR-10 with a CNN the shared base still separates calibrated AUC 0.736 and a 38.4% backdoor lift from 0.561 and 7.6%.
In a character-level transformer language model, random character strings make secrets that appear only in the private fine-tuning text as extractable from the attacker's copy as from the private model itself. Earlier extraction studies queried teachers with natural text resembling the private records. On Tiny Shakespeare the 20 six-digit secrets reach member exposure 19.36 bits in the attacker's copy against 19.65 bits in the private model and about 1.3 bits for non-members, while the sibling-base control reaches only 10.76 bits and fails to recover the fine-tuned utility (validation loss 1.941 versus 1.326).
Perspective
The results cover two-hidden-layer MLPs and small CNNs on MNIST, Fashion-MNIST and CIFAR-10, plus a character-level transformer on Tiny Shakespeare, with three seeds per configuration and independent-initialization or sibling-base controls. The auxiliary-output channel is strongest when the teacher stays near its initialization and carries little membership signal for the CNNs and heavily trained teachers within the given attack budget. Differentially private teachers and sufficiently large initialization offsets are tested as mitigations. The language-model part uses a small character-level model, and scaling to large pre-trained transformers fine-tuned with parameter-efficient methods such as LoRA is named as future work.
The theory is exact only in the lazy-training regime, which the experiments verify approximately rather than assume. The weak CNN leakage is traced to poor activation diversity induced by isotropic noise, but architecture-matched query distributions were not designed, so the CNN numbers should be read as lower bounds on what an adaptive attacker can obtain. On CIFAR-10 the full-logit noise attack recovers a membership signal and the backdoor but not the memorized labels themselves. Canaries are inserted four times to obtain a measurable teacher-side signal, so single-occurrence records leak less in absolute terms. Whether attribute inference and data reconstruction travel the same channel remains open.