# MLP Linearization

Instead of relying only on polynomial random features, MLP linearization now approximates activations with low-degree rational functions and splits each layer, via partial fractions, into separable weight-side and input-side inner products. Across synthetic fitting, vision, 3D and language models this route gives lower error at the same feature budget; compressing Qwen3-8B by replacing its last eight feed-forward layers removes 1.17B parameters (14% of the model) and raises decoding throughput from 763.3 to 809.6 tokens/s, with reasoning and knowledge benchmarks roughly at parity with the uncompressed model.

Instead of relying only on polynomial random features, MLP linearization now approximates activations with low-degree rational functions and splits each layer, via partial fractions, into separable weight-side and input-side inner products. Across synthetic fitting, vision, 3D and language models this route gives lower error at the same feature budget; compressing Qwen3-8B by replacing its last eight feed-forward layers removes 1.17B parameters (14% of the model) and raises decoding throughput from 763.3 to 809.6 tokens/s, with reasoning and knowledge benchmarks roughly at parity with the uncompressed model.