ConwayNet adds a fixed Game-of-Life rule to every layer, and the shallow CNN gains double digits under noise
Synopsis
ConwayNet adds to each neural-network layer a fixed, Game-of-Life-inspired lateral-inhibition rule whose parameters are never optimised: it reads the layer input's neighbourhood occupancy and blends birth and survival gates into the output according to each unit's own state. Wired into fully connected, convolutional, wide-residual and graph convolutional networks, it makes activation maps coherent across space, depth and seeds and pushes gradient saliency onto the object; on the shallow convolutional network, accuracy under Gaussian, shot and impulse noise rises by double digits, at a clean-accuracy cost of about one point on MNIST, EMNIST and their colour versions and a few points on FashionMNIST and KMNIST.
Interpretation
Each layer's output is now regulated by that layer's own input through a fixed rule that carries no trainable parameters, so the optimiser cannot rewrite it and must learn to work within it. Previously a network layer was only a non-local linear map plus a pointwise nonlinearity, which left hidden layers hard to read and interpretation largely post hoc. On the ten-layer MNIST multilayer perceptron, the activation-map low-frequency power ratio rises from 0.090 without the rule to 0.941 with it, while the ten stacked hidden layers keep the shape of the input digit.
The rule compares each unit's neighbourhood occupancy with a birth and a survival set-point, uses smooth gates to decide whether the unit is excited or suppressed, and blends the two continuously by the unit's own state. The discrete Game of Life reproduces as a standalone automaton only at steep slopes, and inside the network each layer applies the rule once, so the implementation uses differentiable continuous gates. The rule is wired into four backbones, a fully connected network, a two-stage convolutional network, WRN-40-2 and a two-layer graph convolutional network, and evaluated on the MNIST family, CIFAR-10 and their corrupted test sets, with corrupted images generated from Gaussian, shot and impulse noise on a nine-severity ladder.
On the shallow convolutional network, accuracy on corrupted test images rises by double digits across the grayscale and colour MNIST family, most on EMNIST. The otherwise identical plain network trained with the same recipe scores clearly lower on the same corrupted images, while holding about one point more clean accuracy on MNIST, EMNIST and their colour versions and a few points more on FashionMNIST and KMNIST. All five seed pairs gain on every dataset, and intervals that resample seed pairs and test images lie far from zero; on the wide residual network the gain is single digits, and on the graph convolutional network it is double digits under Gaussian and impulse noise but negative under shot noise.
Perspective
The demonstrated setting is image classification on the MNIST family and CIFAR-10 and their corrupted versions, with a ten-layer fully connected network, a two-stage convolutional network, WRN-40-2 and a graph convolutional network; the authors state that residual and attention backbones need additional optimisation care and were not evaluated here.
The authors list unresolved caveats: networks with the rule train more slowly, often need longer schedules and give up up to a few points of clean accuracy; not every rule trains; the rule is sensitive to input normalisation, and other normalisations, colour backgrounds or inverted contrast can switch its effect off or reverse it; and the gains depend on backbone and data, with some coherence properties of the multilayer perceptron not carrying over to convolutional backbones.