Spiking networks price glutamate above GABA in the loss: inhibitory firing is pruned first and accuracy rises 8.6 points
Synopsis
Trainable spiking neural networks have treated energy as a generic penalty blind to synapse type, even though cortical glutamatergic transmission costs substantially more ATP than GABAergic transmission. Embedding that cost asymmetry as a Lagrangian penalty inside surrogate-gradient training lets a single scalar penalty exert stronger gradient pressure on excitatory pathways in a Dale's-law-compliant recurrent microcircuit. On the Spiking Heidelberg Digits benchmark, the unpenalised network settled at 70.9% in an inhibition-dominated regime, while mild metabolic pressure raised accuracy to 79.5% and cut energy from 86,519 to 16,019 ATP units per sample.
Interpretation
The metabolic penalty acts as a structural regulariser rather than a mere energy cap: mild pressure moves the network out of an inhibition-dominated, inefficient regime, improving accuracy and energy together. Without the penalty the network converged to 70.9% validation accuracy, inhibitory firing settled at 0.70 spikes per step, I→E weights grew over training, and the population code stayed low-dimensional. Training at fixed recurrent density 0.10 across 16 penalty strengths, peak accuracy reached 79.5% at penalty strength 10^-5, 8.6 percentage points above the unpenalised baseline; at that operating point energy per sample fell from 86,519 to 16,019 ATP units, about 5.4-fold.
Beyond the productive band the network collapses, with accuracy and energy both heading toward zero as the recurrent circuit stops spiking. At the knee penalty strength of 5×10^-6 used for the two-dimensional sweep, accuracy was 78.8% with a 3.5-fold energy reduction, placing high accuracy and large savings together near the knee. At penalty strength 10^-3 the circuit silences completely; information efficiency in bits per ATP peaks inside the productive band at an operating point where accuracy is still 75.1%.
The single cost asymmetry also produces an emergent pruning order by pathway, eliminating excitatory connections first while inhibitory connections resist longest. With no pathway-specific prior, all four recurrent pathways (E→E, E→I, I→E, I→I) plus the input projection and the readout follow one identical gradient rule. At penalty strength 10^-6, E→E was 71.3% pruned, E→I 67.0% and input 49.9%, while I→E was only 23.0% pruned and I→I 54.0%; by 10^-3, E→E reached 100% and I→E 64.0%.
These structural motifs hold across a two-dimensional sweep of cost ratio and structural density, with density the dominant axis of quantitative variation. The sweep crossed five glutamate/GABA cost ratios (1.0 to 2.0) with fifteen initial recurrent densities (0.05 to 1.0), 75 conditions in all, at fixed penalty strength 5×10^-6. Across the 75 conditions validation accuracy ranged from 64.9% to 79.2%, mean 74.3%±3.9%; E→E pruning stayed between 82.2% and 92.5%, and I→E between 38.9% and 73.9%.
Perspective
The conclusions come from simulated training on Spiking Heidelberg Digits, a 20-class English and German spoken-digit benchmark, using a recurrent microcircuit of 800 excitatory and 200 inhibitory leaky integrate-and-fire neurons, with metrics averaged over the final 5 epochs and across 3 random seeds. The authors propose scaling the microcircuit to larger networks with multiple interneuron subtypes (PV, SST, VIP), or training on tasks that explicitly require disinhibitory control, as the next step for testing whether the elimination of inhibition is general.
The authors note that beyond the productive band accuracy falls steeply over a narrow range of penalty strength, and that learning-rate schedules, gradient clipping or adaptive penalty strategies might smooth this transition. The energy model is a first-order approximation that omits presynaptic vesicle depletion and recycling, astrocytic spatial constraints, and the nonlinear relation between firing rate and oxidative ATP demand; penalising weight magnitude alone cannot remove zombie synapses whose presynaptic neurons have fallen silent, which would need an activity-independent maintenance cost. Training uses BPTT with surrogate gradients as a normative optimisation tool rather than a biological learning rule, and whether biologically local rules converge to the same solutions remains open.