Guy Hadad

A GNN weight matrix acts on graph-propagated features, so an update that is well normalized in parameter space can still be amplified by graph structure into a large representation-level perturbation. GraphMuon moves Muon's matrix orthogonalization into a metric defined by the covariance of the propagated layer inputs and scales each step by its induced effect on the layer representation. On three large-scale OGB benchmarks it reaches AdamW's validation-loss target in 1/2.32 to 1/1.32 of the wall-clock time, a 1.32x to 2.42x speedup.

A GNN weight matrix acts on graph-propagated features, so an update that is well normalized in parameter space can still be amplified by graph structure into a large representation-level perturbation. GraphMuon moves Muon's matrix orthogonalization into a metric defined by the covariance of the propagated layer inputs and scales each step by its induced effect on the layer representation. On three large-scale OGB benchmarks it reaches AdamW's validation-loss target in 1/2.32 to 1/1.32 of the wall-clock time, a 1.32x to 2.42x speedup.