Gating each ReLU feature with a partition-of-unity weight stops regression spectral bias from growing with window count, but on Poisson it makes the lowest frequency the slowest mode

Multiplying each ReLU feature by a hat partition-of-unity weight yields a gated local dictionary that is still a shallow network and still linear in its trainable parameters, while its Gram matrix becomes block-tridiagonal for any local bilinear form. The earlier global ReLU dictionary has a Gram matrix close to a finite-element mass matrix on a uniform grid, with a condition number that grows quartically in the number of features, which is the source of the strong spectral bias of shallow ReLU networks. In regression experiments at matched parameter count, the global dictionary's frequency-dependent rise times span five decades while the gated dictionary stays within a factor of five; the gated mass-matrix condition number barely moves over the tested sizes, and values at the largest size agree with the analytic limits.

The block-tridiagonal interior is a block-Toeplitz matrix whose matrix-valued symbol links architecture to training dynamics: read at the frequencies the gate grid supports, it gives a learning rate per frequency, and the ratio of fastest to slowest rate is the spectral-bias factor. Neural-tangent-kernel analyses describe a fixed kernel and give limited guidance on how a finite-width architecture should be modified; a finite-dimensional Gram-matrix view instead attributes the bias to a structured matrix symbol the architecture controls directly. In the regression case the symbol has a positive lower bound and the spectral-bias factor is uniformly bounded in the number of windows; in the stiffness case the symbol vanishes at zero frequency, the spectral-bias factor grows quadratically with model size, and the frequency ordering is inverted so the lowest frequency becomes the slowest mode.

The same symbol also yields a preconditioner: a block-Toeplitz preconditioner built from the interior stencil removes the frequency dependence of the learning rates exactly, leaving only a boundary effect. Gating cannot change the fact that the stiffness symbol vanishes at zero frequency, and changing the gate profile or knot pattern does not help, because the cause belongs to the bilinear form rather than to the dictionary. On the Poisson problem the preconditioner lowers the gated stiffness matrix's measured spread, shortens rise times by roughly a factor of two and a half, and flattens the curve across the band; with random-phase targets the unpreconditioned spread varies widely over five seeds, while the preconditioned one stays within a narrower range for every seed.

The analysis targets one-dimensional shallow ReLU models linear in their trainable parameters, with a fixed dictionary, and conclusions come from the block-Toeplitz symbol; the regression conclusion is reproduced empirically when hidden-layer thresholds are trained as well, but there the Gram matrix evolves along the trajectory and the analytic prediction no longer applies directly.

The boundary treatment is not entirely rigorous, and a more accurate variant would sharpen the boundary-layer terms of the decomposition without changing the interior symbol or its consequences; the bridge from trainable thresholds to a preconditioned output layer was not extended to the stiffness case, where coupling fast and slow dynamics was not found robust; extending the analysis to deep architectures would leave the symbol no longer fixed.