# Newton–Schulz Orthogonalization

A GNN weight matrix acts on graph-propagated features, so an update that is well normalized in parameter space can still be amplified by graph structure into a large representation-level perturbation. GraphMuon moves Muon's matrix orthogonalization into a metric defined by the covariance of the propagated layer inputs and scales each step by its induced effect on the layer representation. On three large-scale OGB benchmarks it reaches AdamW's validation-loss target in 1/2.32 to 1/1.32 of the wall-clock time, a 1.32x to 2.42x speedup.

A GNN weight matrix acts on graph-propagated features, so an update that is well normalized in parameter space can still be amplified by graph structure into a large representation-level perturbation. GraphMuon moves Muon's matrix orthogonalization into a metric defined by the covariance of the propagated layer inputs and scales each step by its induced effect on the layer representation. On three large-scale OGB benchmarks it reaches AdamW's validation-loss target in 1/2.32 to 1/1.32 of the wall-clock time, a 1.32x to 2.42x speedup.

For Muon under momentum accumulation and a tuned finite-step Newton–Schulz update, the work proves that full-batch training of a sufficiently wide two-layer ReLU network reaches any target empirical squared loss ε with high probability when the constant learning rate scales as (1-μ)√ε, with a hitting-time bound of O((1-μ)^-1ε^-1/2) and a sufficient width independent of both target accuracy and momentum.

For Muon under momentum accumulation and a tuned finite-step Newton–Schulz update, the work proves that full-batch training of a sufficiently wide two-layer ReLU network reaches any target empirical squared loss ε with high probability when the constant learning rate scales as (1-μ)√ε, with a hitting-time bound of O((1-μ)^-1ε^-1/2) and a sufficient width independent of both target accuracy and momentum.