# AdamMe

AdamW's fixed points coincide with the stationary points of a constrained, regularized implicit objective, yet those points need not be stable under AdamW's own dynamics. Working in the continuous-time limit, the analysis gives exact one-dimensional stability conditions and sufficient higher-dimensional ones, with examples where even a strongly convex objective loses stability and, in the common momentum regime, trajectories oscillate persistently instead of converging. Rescaling weight decay by the strength of the gradient update yields AdamMe, which keeps the same fixed points and admits a Lyapunov function guaranteeing that bounded trajectories converge.

AdamW's fixed points coincide with the stationary points of a constrained, regularized implicit objective, yet those points need not be stable under AdamW's own dynamics. Working in the continuous-time limit, the analysis gives exact one-dimensional stability conditions and sufficient higher-dimensional ones, with examples where even a strongly convex objective loses stability and, in the common momentum regime, trajectories oscillate persistently instead of converging. Rescaling weight decay by the strength of the gradient update yields AdamMe, which keeps the same fixed points and admits a Lyapunov function guaranteeing that bounded trajectories converge.