Swarm IntelligenceSource publication:
Loading content…
When honeybees choose a nest site, individuals blindly switch to the option advertised most strongly by neighbors, and this local imitation is shown to be equivalent, at the group level, to a single online reinforcement-learning agent solving a multi-armed bandit. That macro-agent updates nest-site probabilities by a rule named Maynard-Cross Learning, with each scout acting as one parallel action-reward sample. The derivation covers the weighted voter model, a simplified description, and simulations indicate roughly 500 scouts suffice to track the macro-dynamics and converge on the best site.
Swarm-level choice can be written as an ordinary multi-armed bandit learning process rather than left as pure emergence: the population's share of individuals per nest site is the agent's action probability, and an individual's sampled site quality is a reward sample. Previously, imitation of success and the weighted voter model were expressed only as replicator dynamics, and individual trial-and-error learning and group imitation were treated as two separate routes to convergence. Matching the Taylor replicator dynamic aggregated from imitation of success against the expected Cross Learning update term by term, both take the same form when the population vector and the policy vector share one notation, which yields the proposition that imitation of success equals exact Cross Learning.
The update aggregated from the weighted voter model is named Maynard-Cross Learning and corresponds to the Maynard-Smith replicator dynamic, so a scout group that only imitates blindly runs a reinforcement-learning algorithm at the macro level. The weighted voter model asks only that an individual broadcast its site with a strength set by its quality estimate and switch to the first type it encounters in its neighborhood; individuals are not modeled as learners and do not compare each other's quality estimates. Under a well-mixed population with neighborhood size much larger than one, one bee's effect on population inflow and outflow is written as a single policy update whose learning rate equals the inverse of swarm size, and summing over all individuals gives the Maynard-Smith replicator dynamic.
The swarm is not merely equivalent to one learning agent but to one sampling many parallel environments, because each scout is an independent action sample from the policy. A single agent learning in a streaming fashion needs a small learning rate to converge on the best option, whereas the swarm uses its scout count as the number of parallel environment copies. In simplified simulations with 10 options, qualities spread between 0 and 1 and estimation noise modeled as a uniform perturbation of amplitude 0.2, about 500 scouts were enough to closely follow the macro-dynamics and converge on the best site across 1000 seeds, while too few scouts often converged on a sub-optimal site.
The result applies to the weighted voter model, a simplified description of honeybee nest-site selection, under a well-mixed population, neighborhood size much larger than one, and individuals that only imitate blindly. The intended readers are researchers working where swarm intelligence, evolutionary game theory and reinforcement learning meet, and designers of swarm robotics. The simulation findings on scout number and neighborhood size are stated to be qualitative rather than quantitative, with arbitrarily chosen parameters.
The weighted voter model is only a partial description of real honeybee nest-site selection, omitting spontaneous commitment, stop signals, direct commitment switches and quorum sensing, so Maynard-Cross Learning is not an exhaustive account of that biological process. Pure imitation can let a type vanish entirely in finite populations and offers no mechanism for continued exploration in non-stationary environments. Spatial structure and individual heterogeneity are also left unmodeled.