Skip to main content
Back to timeline
arXivSource publication:

A frozen vision model reads EEG directly: SeeEEG learns only how to display signals and beats EEG foundation models on all eight datasets

Synopsis

EEG foundation models transfer poorly as frozen feature extractors, so SeeEEG takes another route: it renders EEG as images using the montage, filtering and contrast settings of a clinical EEG viewer and trains only those display parameters, leaving the vision backbone frozen. With a frozen DINOv3 ViT-B as encoder, SeeEEG exceeds seven EEG foundation models in balanced accuracy on all eight clinical, sleep, affective and brain-computer interface datasets, reaching a mean rank of 1.3 by Cohen or AUROC against 2.9 for the strongest baseline, REVE.

Source-provided article image: Your Vision Foundation Model Is Secretly an EEG Foundation Model
arXiv

Interpretation

A vision foundation model that has never seen EEG can serve as a stronger frozen EEG encoder than specialized EEG foundation models, provided each channel is rendered into its own row of image patches. EEG foundation models are pretrained on thousands of hours of recordings, yet their frozen features transfer poorly to many downstream tasks and do not reliably improve with larger models or more pretraining data. Across eight datasets with every encoder frozen and paired with identical readout heads, SeeEEG with a frozen DINOv3 ViT-B outperforms all seven EEG foundation models in balanced accuracy and reaches a mean rank of 1.3 by Cohen or AUROC, versus 2.9 for the strongest baseline, REVE.

SeeEEG does not train the encoder; it learns how EEG is presented to the vision backbone, with each channel occupying one row of patches drawn either as a fixed-gain waveform or as a scalogram whose frequency range spans the patch height. Earlier image-based EEG routes either still required training or distillation on EEG, or processed each channel as a separate image, so spatial relationships across electrodes could not appear within one image. The display module is built from the montage, filtering and contrast parameters of a clinical EEG viewer, initialized from a standard clinical montage and optimized with the task loss through the frozen backbone, with at most 1,113 trainable parameters per view; evaluation spans eight datasets covering event classification, abnormality detection, sleep staging, dementia, slowing, emotion, imagined speech and motor imagery.

The readability comes from natural-image pretraining rather than from any ViT being able to read rendered EEG. Replacing DINOv3 with a randomly initialized ViT-B lowers the score by 15.2 points on average with the same display and readout heads, and by up to 36.6 points on slowing classification. The learned displays transfer without further optimization to the video encoder V-JEPA 2.1, which outperforms every EEG foundation model on both metrics on six of the eight datasets; a supervised ImageNet-21k ViT does so on four.

Perspective

Evaluation covers eight datasets with all windows sampled at 200 Hz, every encoder frozen and paired with identical readout heads, averaged over three seeds on the test split; this scope supports conclusions for the tested clinical, sleep, affective and brain-computer interface tasks rather than all EEG settings.

Display parameters are optimized against cross-fitted readout heads to avoid fitting subject-specific traits or a particular head, and validation selects among clinical montages, closed-form displays and the learned display, so the reported advantage depends on that validation selection procedure.