Neural networks now build their own learning machinery: random units self-organize by simple rules and match backprop-trained nets on image classification

An artificial neural network can build its own learning machinery instead of having its connections and learning rules designed in advance by a developer. The prior way to judge a network's value to neuroscience was how well it performed a task, with its structure set by an external designer. Researchers at MIT's McGovern Institute and the K. Lisa Yang Integrative Computational Neuroscience Center let a group of randomly connected units reshape itself under simple rules until it could recognize patterns and classify images.

The network starts from multiple layers of randomly interconnected neuron-like units, and within each layer every unit follows the same rules for adjusting connections, so the units interact only through those rules and develop behavior. Those rules match the simple rules thought to rewire brain circuits as humans and other animals learn, such as neurons that fire together wiring together. Learning is carried by self-assembling motifs that emerge under the simple rules, then serve as building blocks for functional circuits and reorganize as the network learns.

The network reorganizes its connections using an optimization strategy similar to the one built into most modern machine learning systems, gradient descent, in which connections are gradually adjusted in response to errors to improve performance. Most modern AI implements gradient descent through some form of back-propagation, which neuroscientists say the brain is unlikely to use. The team compared its self-organizing network with networks designed to learn by back-propagation and found it performed at a similar level.

The work speaks to neuroscientists asking how the brain learns and to engineers looking for network design ideas in biological rules. The authors call self-organization the first high-level framework, saying that once self-organization works as a measure, the next thing can be discussed. They also note that knowing the rules behind neuron-neuron interactions is a precondition for applications such as brain-computer interfaces.

The authors state that a self-organizing network does not necessarily learn the same way the brain does, and that it is a critical starting point for biological plausibility. They position self-organization as the first high-level framework rather than an endpoint. They also say a strong connection between machines and brains would follow only if the proposed synaptic motifs are in fact what the brain uses to do what backpropagation does for artificial networks, a condition that remains a hypothesis.