Population Based Methods

“One general law, leading to the advancement of all organic beings, namely, multiply, vary, let the strongest live and the weakest die. – Charles Darwin”

• Genetic Algorithm: suitable for discrete solution space.

• Differential Evolution: similar as Genetic Algorithm, except that 1. using continuous variable instead of discrete. 2. reproduction involves combining the parameters of 3 individuals instead of 2.

• Evolution Strategy: Similar to DE, except that it adds noise instead of mutation during offspring.

• Covariance Matrix Adaptation Evolution Strategy: Instead of just add noise in ES, CMAES uses the mean and covariance matrix of the search distribution to generate population for the new back.

• Particle Swarm Optimization: inspired by particle movement, use position and momentum of particle for the selection of population in the new generation.

• Cross-Entropy Method: update the probability distribution using fitness of samples, and use the updated probability distribution for the next sampling.