Memorability models flip their scores under noise, while GraphMem tracks human judgments by reading how images relate
Synopsis
After progressive Gaussian noise was added to LaMem images, mainstream memorability models such as ResMem, ViTMem and AMNet estimated lower memorability for initially memorable images and higher memorability for initially forgettable ones, a reversal resembling stochastic resonance. In a continuous recognition task with 88 participants, memorable images proved more resistant to noise while forgettable images declined sharply. The authors built GraphMem, which estimates memorability from an image's neighborhood in a similarity graph rather than from single-image features; it aligned most closely with human data under noise and blur, and after noise training its slope effect reached ρ=0.57 (95% bootstrap CI [0.29, 0.75]), the only model to turn positive.
Interpretation
Mainstream memorability models estimate memorability in the opposite direction from human observers when images are degraded by noise. These models had been validated against human memorability almost exclusively on clean images, and human performance under noise had not been compared on the same image set. On 980 images from 196 LaMem scenes across seven memorability bins and five Gaussian noise levels, ResMem's memorability slopes varied significantly by bin (F(6,189)=18.68, p<.001, η²p=0.37), and ViTMem and AMNet reproduced the same direction.
Human recognition worsened overall under high noise, but initially memorable images stayed relatively stable, and the divergence was driven mainly by false alarms. Models predicted that forgettable images would become more memorable under noise, whereas human data showed forgettable images were more often mistaken for repeats. In the continuous recognition task with 88 participants, the fixed effect of bin on corrected recognition was β=-0.17 (SE=0.05, t=3.36, p<.001), false alarm rate varied significantly by bin (F(6,522)=4.60, p<.001), and hit rate showed no significant bin effect (F(6,522)=1.18, p=.315).
GraphMem estimates memorability from an image's position in a similarity graph rather than from features of the image alone. Conventional models look at single images and therefore cannot reflect how noise changes similarity relations among images; pixel-level typicality analysis showed noise generally lowered typicality, more so for low-memorability images (β=-0.06, SE=0.02, t=3.86, p<.001). GraphMem builds a k=10 nearest-neighbor graph from 768-dimensional ViT-B/16 embeddings and passes it through a two-layer graph convolutional network to output a memorability score; it had the highest Spearman correlation with human data under noise and blur, and after noise training its slope effect was ρ=0.57 (95% bootstrap CI [0.29, 0.75]), while other models ranged from -0.54 to -0.82.
Perspective
The conclusions rest on 196 LaMem scene images and their five Gaussian noise and five blur levels, with human data from an online continuous recognition task run with George Mason University undergraduates; GraphMem aligned best with human data under noise and blur but was overall worse than conventional models at predicting memorability for individual images.
Human data did not show the stochastic-resonance-like reversal seen in models; the authors suggest this may reflect differences in how humans and models separate signal from noise, but whether memorability has a detection-threshold-like mechanism is unclear. Noise training did not reliably improve model prediction of LaMem memorability or human recognition, and some models performed worse.