Polytopal Neural Networks constrain latent representations to learned polytopes, retaining 96.9% of unconstrained accuracy on ImageNet-1k
The authors propose Polytopal Neural Networks (PNNs), which constrain each layer's latent representation during training to a convex combination of learned archetypes so that the explanatory coordinates are the same coordinates the next layer receives; across image classification and reconstruction benchmarks, PNNs approach unconstrained networks with small accuracy loss, outperform VQ-VAE and Dirichlet VAE in unsupervised reconstruction, and offer a VQ training route without EMA updates, commitment loss, or straight-through estimator.