How much of a conv layer's own input survives its output: compression beats depth as a predictor, and copying pretrained singular vectors raises recoverability
Across nine ImageNet convolutional networks, how much of a layer's own input patches can be reconstructed from its output falls with depth but is more consistently predicted by the layer's effective compression, and copying pretrained singular vectors onto a random encoder raises recoverability while copying the pretrained singular-value spectrum or activation sparsity lowers it.