Last-layer weights alone can rank model quality: a statistical account of heavy-tailed spectra, with an overfitting marker during training
A Bayes free energy computed from trained final-layer weights alone tracks model quality across convolutional networks and LLMs, and under a heavy-tailed spectral model it yields an optimal heaviness whose crossing marks the onset of overfitting.