Stochastic Methods

“One doesn’t discover new lands without losing sight of the shore. – Andre Gide” sailing

  • Random Search: a brute-force method for solving optimization problems. It randomly generates a set of parameters and evaluate the quality of the solution, it then pick the best one among them

  • Monte Carlo method: it uses a large number of random sampling to approximate a function or a probability distribution

  • Simulated Annealing: The algorithm starts with a random solution and iteratively generates new solutions by making small random changes to the current solution. It then decides whether to accept the new solution or not based on its quality and the current temperature. The temperature is gradually decreased over time, which causes the algorithm to become more selective in accepting new solutions. This simulates the cooling process of annealing in metallurgy and helps the algorithm escape local optima and converge to the global optimum.