A cortical implant gets its encoder: Braincodec restores natural sound on 256 channels, and mice discriminate its patterns as sharply as real tones

The auditory cortex has more than ten times the surface area of the cochlea or auditory brainstem and can host hundreds of stimulation channels, but no encoding algorithm converted sound into cortical stimulation patterns without losing acoustic information. Cochlear and auditory brainstem implants are limited by the spatial resolution of electrical stimulation, offering few independent channels: enough for speech in quiet, but not for noisy backgrounds or musical detail. Across 2000 sounds, Braincodec reconstructions averaged an SI-SNR near +10, while the 22-channel ACE cochlear-implant encoder, decoded by the same class of deep decoder, averaged -33.1 plus or minus 8.4, with paired two-sided t-tests giving p below 0.001.

Braincodec trains a 14-layer one-dimensional convolutional encoder, a mirrored decoder and a shallow classifier together, with the encoder turning a 24 kHz waveform into 40 Hz channel stimulation patterns. An autoencoder alone preserves acoustic information but poorly reproduces cortical representations, while a classifier alone approaches cortex but cannot reconstruct sound. On fMRI responses to 165 natural sounds in human auditory cortex, Braincodec representations explained up to 71 percent of primary auditory cortex activity, against no more than 19 percent for the compression-only autoencoder, and the two training objectives did not degrade each other.

In implanted mice, 100-channel Braincodec patterns were delivered through a 10 by 10 microLED array over primary auditory cortex field A1, and the mice learned a Go/NoGo discrimination between two patterns. Non-implanted mice instead discriminated two real pure tones at 4 and 16 kHz, and the two groups showed no significant difference in learning curves (p equals 0.56). In catch trials with intermediate frequencies, perceptual resolution was 0.21 plus or minus 0.02 octave for the Braincodec group and 0.26 plus or minus 0.02 octave for the real-tone group, a non-significant difference (p equals 0.11).

The demonstrated scope is an offline-trained, online-running software encoder plus an optogenetic stimulation prototype over mouse A1; human data serve only for representational similarity, with no human implant trial. The authors propose validating high-resolution surface electrodes for human stimulation as the next step.

The authors note that Braincodec aligns its output only to the mesoscopic organization of cortical sound responses and cannot reproduce the exact activity pattern at single-neuron scale, so implant effectiveness will depend on the patient brain's ability to associate the new representations with meaning. The mouse experiments transposed pure-tone patterns from the human hearing range onto the mouse tonotopic axis, and natural sounds were not used in that experiment.