Cross-subject brain-image retrieval breaks on new people: SA-LBC anchors calibration to same-stimulus source responses, lifting MRR from 0.1867 to 0.2336
Synopsis
In cross-subject neural-image retrieval, comparing the mean of a few target observations with the global source mean conflates subject-specific representation shifts with differences in which stimuli were sampled. SA-LBC instead estimates a first-order shift by averaging residuals against same-stimulus source-population anchors, then shrinks the correction using source-derived uncertainty. On THINGS-EEG2 with 10 subjects under leave-one-subject-out evaluation, the Attention-CLIP ViT-B/32 configuration rises from 0.1867 MRR without calibration to 0.2013 and 0.2336, though retrieval gains remain representation-dependent.
Interpretation
Matching a new subject's brain activity to images no longer relies on a single global source mean; it uses the source population's average response to the same stimuli as the reference for estimating that subject's latent shift. Global-mean matching subtracted the overall source centroid from the target calibration mean, so the difference mixed subject mismatch with differences in sampled stimulus composition. In the 10-subject leave-one-subject-out evaluation on THINGS-EEG2, the Attention-CLIP ViT-B/32 configuration rises from 0.1867 MRR without calibration to 0.2013 and 0.2336; across all six neural-visual configurations and all calibration budgets, same-stimulus anchors yield higher MRR than anchors built with shuffled stimulus identities.
The calibration is first averaged from same-stimulus residuals, then discounted by a reliability factor estimated from source data, so poorly supported shifts are not applied at full strength. The unshrunk anchored estimate can fall below no calibration at the smallest budget, for example 0.1822 versus 0.1867 for Attention-CLIP ViT-B/32 at M=2. At M=2, shrinkage improves on the unshrunk anchored estimator in all six configurations; on THINGS-EEG2, mean relative discrepancies between analytic and empirical MSE are 0.90-1.61% for anchoring and 1.05-1.58% for global-mean estimation.
Perspective
Evaluation is limited to fixed preprocessing and a frozen source-trained representation, primarily the 10-subject leave-one-subject-out setting on THINGS-EEG2, with descriptive results on 20 subjects from Alljoined-1.6M and four subjects from THINGS-MEG; calibration uses only a small set of stimulus-matched observations that are concept-disjoint from the test set.
Retrieval gains vary with representation, and SA-LBC's effect over no calibration is small or negative on TSConv configurations; shrinkage behavior at larger budgets is not uniform, and the ordering of geometric corrections can reverse with regularization strength; gains on Alljoined-1.6M are smaller and heterogeneous, and THINGS-MEG covers only four subjects, making that evidence descriptive.