Neuroevolution changes the weights or the shape of a neural network by evolution instead of by training it.
A genetic algorithm looks for the weights that give the best score on the task.
Other ways of training a network need a clear goal and a way to measure the difference between the answer and the right answer. Neuroevolution does not, which is why it works on problems where you cannot say what a good answer looks like.
What is Neuroevolution?
You can also look at it as a search: try a lot of networks, keep the ones that score well, change them a bit and try again.
Short version: an evolutionary algorithm builds a neural network for you, the same way plants change over generations.
Because it only needs a score, not a gradient, you can use it with any network and with tasks where you only know whether the behaviour was good or bad.
It is used most in robotics and in artificial life research.
You can also use it to study how simple behaviours arise in nature, by evolving them and seeing what comes out.
The goal is a network that behaves well without anybody writing the behaviour down. You can also use the same code to look at how learning works in nature.
Applications of Neuroevolution
So it suits problems where there is no right answer to copy. The clearest use is robot controllers.
Neural controllers have been evolved to drive robots, cars and rockets. The work goes back to the early 90s.
The same approach has been used for design problems, for example to grow structures or to design circuits.
The catch is that the controller has to be built in a simulation and then moved to the real robot, which rarely works first time.
It is also good to run in parallel, because you can try many candidate networks at the same time. And it has been used to write game characters that learn instead of following a script.
In games the non player characters are usually scripted. Evolve them instead and you get behaviour nobody wrote.
That opens up games where the player trains a group of agents. You can also evolve the weapons, or the vehicles, or the opponents.
It is also a research tool: you can evolve a network and then look at what it came up with to learn something about the problem.
You can use it to ask a question about biology, for example which conditions are needed for a behaviour to appear.
The same code works on simple tasks like foraging, avoiding danger and following each other, and you can see whether the behaviour appears out of nothing.
You can also use it to explore a changing environment, where the rules shift while the network is running.
The fastest way to learn this is to write it yourself. The PyChallenge exercises run in your browser, no install needed.