Goodfire reads Qwen3.6 neurons directly: ablating 92.2% of weak activations raises language-model loss by only 2.9%

An individual MLP neuron can be studied directly as an interpretable unit of computation, without first transforming the model into a learned feature space. Neurons had long been treated as too complex and polysemantic for direct study, which pushed much of the field toward learned feature spaces over sets of neurons. In Qwen3.6-35B-A3B, keeping only the 320 strongest router-weighted neurons per forward pass and ablating 92.2% of weak activations raised language-model loss by 2.9%, with non-reasoning benchmarks close to the intact model.

Each neuron pole has few computational functions, usually a single mode or one mode that dominates causally. Polysemanticity was understood as a single neuron responding to many seemingly unrelated concepts. Across a sample of 2,734 neurons, 47% of poles had only one mode, and about half of multimodal poles had one mode accounting for at least 90% of their KL damage.

A natural-language description of a neuron is enough to predict, disambiguate and simulate its firing. Earlier neuron descriptions were largely observational or correlational and did not support causal prediction under intervention. Above p97/3, a judge model given the description picked the firing snippet 95% of the time against random-text distractors (chance 50%); in the strongest activation band, description-based simulation recovered 73% of ablation damage relative to a perfect-reader ceiling.

The findings cover the mixture-of-experts MLP neurons of Qwen3.6-35B-A3B, focus on the p97/3 and stronger activation regime, and leave shared-expert neurons unchanged in every sparsification experiment. Released alongside are circuit diagrams, activation examples and description pages for a stratified set of 2,734 neurons, plus low-effort labels for all ten million neuron poles.

The role of weak and Q3 activations is not yet analyzed and is left to future work; mode counts depend on descriptive granularity and are not definitive counts of a neuron's concepts; the contextual-reuse mechanism is shown in examples, without establishing how prevalent it is or how reliably it resolves ambiguity; how far these findings generalize to other models and behaviors remains to be established.