Brain maps now decode to tasks, concepts and domains at once, not just a task name

A brain activation map can now be decoded into cognitive labels at three levels, task, concept and domain, rather than a task name alone. Earlier functional decoding mainly produced task-level predictions; correlation-type decoders depend on precomputed meta-analytic maps and fixed vocabularies, and GC-LDA operates entirely unsupervised and is not optimized for classification or prediction accuracy. On group-level maps for seven Human Connectome Project task fMRI domains (emotion, gambling, language, motor, relational, social, working memory), the best configuration reached 62.86% task recall, 43.57% concept recall and 90.48% domain recall, while Neurosynth and GC-LDA reached at most 20.71% task recall.

Decoding accuracy depends on whether training text is the full article or the abstract, and on whether the ontology is curated. Training on abstracts alone gave lower text-to-brain association scores for every language model, and the full Cognitive Atlas ontology produced minimal or zero recall in some configurations. With full body text and BrainGPT-7B-v0.2, Recall@10 was 33.56±4.12, Recall@100 71.95±4.18 and Mix&Match 90.04±1.66, against 24.01±1.84, 61.98±4.28 and 85.82±1.84 for the same model on abstracts; the reduced curated ontology outperformed the original across all models.

The model converts text-to-brain similarity into probabilities that propagate through the ontology to yield concept and domain outputs. Contrastive learning had mainly served text-to-brain association retrieval, without systematic evaluation of brain-to-text decoding or mapping onto a structured cognitive vocabulary. Text embeddings come from a pretrained large language model, brain maps from activation coordinates modeled with MKDA and reduced by the DiFuMo 512 parcellation, and both pass through CLIP encoders into a shared latent space; cosine similarity becomes a likelihood through softmax, the posterior combines it with a task prior from the literature, and concept and domain probabilities aggregate along task-to-concept and concept-to-domain mappings via a noisy-OR model.

Across six anatomical regions of interest, predictions show clear regional selectivity. Compared with dense group-level maps, the gap between the top predicted task and the second-highest is more distinct for regions of interest. The right temporoparietal junction predicted social cognition/theory of mind tasks at 98.5%, the amygdala associated with emotion processing at 65.5%, the insula with pain monitoring/discrimination at 34.9% mapping to the emotion domain at 76.9%, and the striatum with semantic tasks at 52.1%.

Validation covers group-level activation maps for seven Human Connectome Project task domains, six meta-analytic anatomical regions of interest, and subject-level maps from 787 participants; the authors explicitly recommend group-level decoding for now and state the model should not be used to decode high-noise subject-level activation maps. Proposed next steps include adding unthresholded statistical maps and subject-level labeled images to training, improving cognitive ontologies, and broader cross-dataset validation on independent task fMRI datasets.

The authors note the training sample of just over twenty thousand pairs is far smaller than typical CLIP training; the model was trained mainly on positively activated coordinates, so decoding of deactivations should be interpreted cautiously; long article text is represented by averaged chunk embeddings that cannot preserve temporal structure; a distributional shift exists between the long training text and the short ontology entries used at inference; task priors are set from literature prevalence and may be suboptimal for rare tasks; and ontology mappings reflect individual researchers' opinions with unstandardized task names and definitions. They also report task recall of 38.19% on subject-level maps, whereas image-based classification methods exceed 90% accuracy on the same individual maps.