Cortico-subcortical multi-head self-attention as a substrate for cognitive performance
The work proposes that cortico-thalamic circuits are well suited to implement multi-head self- and cross-attention: layer 2/3 pyramidal cells maintain a recurrent key-value memory while layer 5 pyramidal cells decode the memory retrieved by an incoming query, the computation of keys, values and queries maps onto core and matrix thalamo-cortical projections distributed across the micro- and macro-columns of a cortical area, one cortical area forms an attention head and cortex a multi-head self-attention network, the same thalamo-cortical microcircuit also calculates sensory prediction errors guiding gradient-based synaptic plasticity, and a reward-prediction error gates via basal ganglia the cortical output and the re-activation of hippocampal memories, with the trained network aligning wit