Gradient descent's implicit bias is no longer set by initialization alone: step size tilts the limit toward a perturbed weighted L1 minimizer
The training limit of deep linear diagonal networks had been characterized only for gradient flow, the continuous version with vanishing step size, where the implicit bias is fixed entirely by initialization. This analysis rewrites discrete gradient descent and stochastic gradient descent themselves as perturbed mirror gradient descent, proves convergence of the iterates for any number of layers, and characterizes the limit as a convex minimization problem parameterized by a perturbed initialization. The perturbation shrinks linearly with step size, so the limit corresponds to weighted L1 minimization whose weights depend on initialization, step size and the whole iterate trajectory.