A frozen EEG foundation model needs no fine-tuning: one first-layer attention head's connectivity matrix reaches 42.13 balanced accuracy in cross-subject emotion recognition
Synopsis
The EEG foundation model REVE no longer needs full fine-tuning or all-layer pooled features to recognize emotion. Such models had not consistently beaten task-specific architectures on cross-subject tasks, and their large representations are costly to compute and store. This approach aggregates the first-layer attention weights of a single head over each trial into a weighted channel-to-channel matrix, then uses a label-free symmetry score to pick the most informative head; on SEED-IV, classical classifiers reach 42.13 balanced accuracy with about 176 times fewer calculations than all-layer attention pooling.
Interpretation
A frozen EEG foundation model can directly yield interpretable channel-to-channel connectivity features instead of feeding pooled layer vectors to a classifier. EEG foundation models had not consistently outperformed task-specific architectures on cross-subject tasks, and the standard use concatenates attention-pooled embeddings from all 23 layers into an 11,776-dimensional feature vector. On the four-class SEED-IV emotion data, the connectivity matrix from a single attention head with classical classifiers reaches 42.13 balanced accuracy and 40.12 weighted F1, in the same range as fully fine-tuned LaBraM at 40.98 and 40.61.
Choosing an attention head needs no labels; the degree of diagonal symmetry in its attention map is enough. Among the eight heads, the third head (index 2) has the lowest Frobenius symmetry error, the highest symmetry score and the smallest standard deviation, and its map shows a more pronounced block-structured diagonal pattern. That head is best on every metric, with accuracy 0.3917 and F1 0.3803, while the other heads sit closer to, though above, the 0.25 chance baseline; permutation testing on that head's features rejects random-feature equivalence at p<0.01.
Feature extraction uses only the first layer, because only there does each token still map to exactly one channel and time window. From the second layer onward, the multi-head attention output projection mixes information across heads and positions, so an individual attention score can no longer be attributed to one channel pair. On the 1,080 SEED-IV recordings, each split into 1-second windows, attention maps are softmax-normalized, stripped of self-connections and pooled over the trial to give one 60×60 matrix per head; this needs 176 times and 8 times fewer calculations than all-layer and final-layer pooling.
Perspective
The demonstrated scope is the SEED-IV dataset: 15 subjects, 3 sessions of 24 trials each, 1,080 recordings in total, 60 channels, a leave-one-subject-out cross-subject protocol, and only REVE's first-layer attention. The authors propose extending the work to window-based weighted connectivity features, other models and datasets beyond SEED-IV.
The authors note that foundation models can be biased by pretraining data and may emphasize neighboring connections, an effect not yet studied for REVE or this method, so the literature on left-right hemisphere asymmetry for positive versus negative stimuli could not be confirmed. The universality of the symmetry-based head-selection criterion and of the frontal-region enrichment still needs other models and datasets.