Skip to main content

# Dense Associative Memory

2 related articles

Subscribe to updates

Nature Communications Oct 25, 2026
Before diffusion models generalize, they grow spurious attractors absent from training data but repeated in samples
As the training set grows, diffusion models move from memorizing training samples to generalizing, and between these phases an intermediate kind of attractor appears that matches no training image yet recurs in generated samples. Reading generation as retrieval in a dense associative memory predicts such spurious states once storage capacity is exceeded; on MNIST, FASHION-MNIST, CIFAR-10 and LSUN-CHURCH the spurious fraction rises and then falls at the memorization-to-generalization boundary, with basin volumes between those of memorized and generalized samples.
As the training set grows, diffusion models move from memorizing training samples to generalizing, and between these phases an intermediate kind of attractor appears that matches no training image yet recurs in generated samples. Reading generation as retrieval in a dense associative memory predicts such spurious states once storage capacity is exceeded; on MNIST, FASHION-MNIST, CIFAR-10 and LSUN-CHURCH the spurious fraction rises and then falls at the memorization-to-generalization boundary, with basin volumes between those of memorized and generalized samples.
Radboud University Nijmegen Rensselaer Polytechnic Institute Sapienza University of Rome CY Cergy Paris Université Tilburg University
Gabriel Raya（First author）Bao Pham（First author）Dmitry Krotov（Corresponding author）
Diffusion Model Dense Associative Memory Memorization-to-Generalization Transition creativity
arXiv Oct 9, 2026
BRACE gives LSR dense associative memory differentially private retrieval, cutting MNIST prediction error from 0.061 to 0.011
The authors propose BRACE, the first differentially private retrieval mechanism for log-sum-ReLU dense associative memory (LSR-DAM): it identifies memory points whose membership changes under privacy noise and applies an adaptive boundary correction before injecting calibrated noise, controlling error accumulation from boundary switching in compact-support retrieval. They prove minimax-optimal terminal and full-trajectory retrieval error rates and establish trajectory central limit theorems characterizing privacy-induced variability; synthetic experiments show a tiny perturbation changes 11 gate memberships, and MNIST retrieval reduces prediction MSE from 0.061 for the corrupted input to 0.011, nearly matching noiseless retrieval.
The authors propose BRACE, the first differentially private retrieval mechanism for log-sum-ReLU dense associative memory (LSR-DAM): it identifies memory points whose membership changes under privacy noise and applies an adaptive boundary correction before injecting calibrated noise, controlling error accumulation from boundary switching in compact-support retrieval. They prove minimax-optimal terminal and full-trajectory retrieval error rates and establish trajectory central limit theorems characterizing privacy-induced variability; synthetic experiments show a tiny perturbation changes 11 gate memberships, and MNIST retrieval reduces prediction MSE from 0.061 for the corrupted input to 0.011, nearly matching noiseless retrieval.
Fudan University University of Ottawa
Chang Qu（First author）
Boundary-Responsive Adaptive Correction Evolution Dense Associative Memory Differential Privacy Memory Retrieval Uncertainty Quantification

You’re all caught up