Gating each ReLU feature with a partition-of-unity weight stops regression spectral bias from growing with window count, but on Poisson it makes the lowest frequency the slowest mode
In shallow ReLU models, the spectral bias that makes high frequencies learn slowly is tied directly to the condition number of the feature dictionary's Gram matrix. Multiplying each ReLU feature by a hat partition-of-unity weight makes that Gram matrix block-tridiagonal for any local bilinear form, and its block-Toeplitz symbol sets the learning rate of each frequency. Under a regression loss the symbol stays away from zero, and the measured spread of frequency-dependent learning rates falls from four orders of magnitude to within a factor of five at matched model size; under the stiffness matrix the symbol vanishes at zero frequency, so gating leaves conditioning intact and instead makes the lowest frequency the slowest mode.