M2N2 evolves neural networks through competition and model fusion
Sakana AI researchers describe a gradient-free evolutionary method that selects and merges models; “reproduction” here is a metaphor for parameter recombination.
Researchers at Sakana AI introduced Model Merging of Natural Niches (M2N2) in the paper “Competition and Attraction Improve Model Fusion.” Instead of conventional backpropagation and gradient descent, the method evaluates a population of neural networks and creates successive generations by selecting and merging suitable models.
How does it work?
M2N2 combines three ideas. It dynamically changes merge boundaries so that different parameter segments from two models can be recombined; it uses competition over data samples to preserve a diverse population; and an “attraction” heuristic pairs models whose strengths complement one another. The description of AI models “reproducing” is therefore a technical metaphor for selection, recombination and mutation—not biological reproduction or autonomous model behavior.
What did the experiments show?
According to the researchers, M2N2 evolved MNIST classifiers from randomly initialized networks without computing gradients, reaching performance comparable to CMA-ES while using computation more efficiently. They also tested the approach on specialized language models and text-to-image models, reporting that merged models improved on target tasks while retaining capabilities that were not directly optimized by the fitness function.
The results suggest that evolutionary model fusion can complement established training and merging techniques. They do not show that AI has acquired autonomous reproduction, nor that the method generally replaces conventional training. The paper was accepted and published at GECCO 2025.