Same architecture, same data, different optimizer: Muon checkpoints reconstruct best, Adam resists most

How much a released checkpoint leaks depends on the implicit bias the optimizer leaves behind, not only on whether the network memorized its training samples. Earlier reconstruction results assumed gradient-descent training, where the attack exploits the maximum-margin implicit bias of gradient descent. On binary and 10-class MNIST and CIFAR-10, one checkpoint-only attack yields different reconstruction strength across models trained with the three optimizers.

The attack writes the stationarity condition of the optimizer's margin geometry and inverts it for candidate inputs. The Euclidean stationarity term of the gradient-descent attack is replaced by a subdifferential condition on the chosen optimizer's margin norm, together with a scale-invariant per-block alignment penalty. The objective recovers the earlier gradient-descent attack up to the multiplier scale in the Euclidean case, and covers both Muon's spectral norm and Adam's global infinity norm.

Adam's stationarity target encodes only the locations and signs of the largest-magnitude coordinates, with no continuous information about weight magnitudes. SGD's target cone is the ray through the weight direction and Muon's encodes the leading singular structure of every block, whereas Adam's depends only on the arg-max set and its signs. At nondegenerate KKT points Adam's stationarity system has at most as many equations as on-margin examples, fewer than the unknowns, so it fails regular first-order identifiability within this attack family.

The report covers checkpoint-only, stationarity-based reconstruction, under a threat model where the adversary sees the final parameters and architecture and knows the optimizer family, but not the training inputs, labels, sample count, or intermediate iterates. Experiments use fully connected ReLU networks on binary MNIST parity, binary CIFAR-10 vehicles versus animals, and 10-class CIFAR-10, with Muon applied only to hidden-to-hidden matrices while input and output layers use SGD. The authors propose extending this geometric implicit-bias analysis to memory-efficient, low-rank, and orthogonalized optimizers as the next direction for understanding training-data leakage.

The authors state no formal privacy guarantee, and the analysis concerns the idealized bias, namely Adam with a negligible stability constant, with the network-level statement resting on a nondegenerate KKT assumption. The Muon-versus-SGD comparison is a prediction from reachability rather than a theorem and shifts with training budget, sample size, and architecture: at small sample sizes SGD becomes comparable to Muon and in some architectures matches or exceeds it. Specificity is not by itself an information-theoretic bound on input identifiability, and Adam's first-order non-identifiability result covers only this stationarity-only attack family.