Skip to main content
Back to timeline
arXivSource publication:

General vision-language models read EEG directly: Qwen3-VL hits 0.835 balanced accuracy on 4-second discharge detection without EEG pretraining

Synopsis

Clinical EEG reading has long depended on experts paging through recordings, while recent EEG foundation models first pretrain on large unlabeled EEG corpora. This work renders EEG segments as the bipolar montage waveforms or spectrograms clinicians already inspect, pairs each image with a fixed text prompt, and hands it to a general vision-language model, Qwen3-VL, with no EEG pretraining at all. On the 4-second interictal epileptiform discharge task vEpiSet, Qwen3-VL-4B adapted with LoRA reaches 0.835 balanced accuracy against 0.763 for fully fine-tuned BIOT; on the TUEV EEG-Bench split, Qwen3-VL-8B reaches 0.750 against 0.739 for the second-best single-task model.

AI-generated editorial illustration: Recasting Visual Language Models as EEG Foundation Models for Clinical Neurodiagnostics

Interpretation

A general vision-language model can take on clinical neurodiagnostic EEG reading without any EEG pretraining, and on some tasks surpasses foundation models built specifically on EEG. Earlier automation either trained a separate supervised model per task or pretrained self-supervised on large unlabeled EEG before fine-tuning; existing EEG foundation models only led in data-rich fine-tuning, did poorly in frozen-backbone linear probing, and showed no performance growth with size. On the 4-second interictal epileptiform discharge dataset vEpiSet, Qwen3-VL-4B adapted with LoRA reaches 0.835 balanced accuracy versus 0.763 for fully fine-tuned BIOT; on the TUEV EEG-Bench split Qwen3-VL-8B reaches 0.750 versus 0.739 for the second-best single-task model, while on the canonical TUEV split Qwen3-VL-4B reaches 0.601, below CS-Brain's 0.690.

The method renders an EEG segment at the task's time scale as an image, so the model answers by looking at a picture much as a clinician does. The input is not raw multichannel signal but a bipolar montage waveform image stacked in the clinical channel order, or a spectrogram or scalogram computed from the same derivations, optionally combining the waveform with one time-frequency view. On vEpiSet the bipolar montage input gives AUPRC 0.849 and AUROC 0.891, above 0.811 and 0.861 for the continuous wavelet transform and 0.702 and 0.809 for the short-time Fourier transform; on SpikeNet2, Qwen-4B at the default 600x896 resolution scores 0.545±0.025 balanced accuracy, near chance, and 0.626±0.024 with 1800x256 signal renderings, close to LaBraM's 0.641±0.028 and CBraMod's 0.654±0.016.

With a frozen backbone and only a classification head trained, the general vision-language model adapts faster than EEG foundation models in few-shot settings. Freezing the backbone and training only a shallow head tests the quality of the pretrained representation itself, rather than letting abundant labels override a weak representation. On TUEV with 64 samples per class, Qwen3-VL-4B reaches 0.464±0.043 balanced accuracy against 0.331±0.076 for the closest competitor; a Qwen3-VL-4B first LoRA-adapted on TUAB reaches 0.406±0.043 with 10±1 examples per class while competitors stay near chance, and 0.521±0.023 with 384 examples, comparable to the 0.528 of BIOT trained on the full TUEV corpus.

Perspective

The experiments focus on EEG tasks in which trained experts are expected to identify the relevant patterns by eye: normal-versus-abnormal classification on TUAB, six-class event recognition on TUEV, interictal epileptiform discharge detection on vEpiSet, and few-shot evaluation on SPaRCNet, SpikeNet2 and SeizeIT2; the data come from existing public or controlled-access EEG repositories, and no new participant-level data were generated.

Whether general pretraining will keep improving EEG task performance remains an open empirical question, and the transferability of general vision features may have natural limits; for applications where the relevant signal information is subtle and not visually apparent even to experts, the benefit of the vision-language route is largely unknown; and the deployment cost of these larger models remains a significant practical barrier to adoption.