Gradient Free

“Is it possible to navigate without a compass?” “Yes, through the use of triangulation, satellite, or a combination of both.” – Yuli evolution

We cannot compute the gradient of the objective functions and use it to guide our optimization, due to reasons such as:

  1. No analytical description of the function (e.g. result is obtained from simulation or experiments).

  2. Stochastic function evaluation (e.g. noisy).

  3. Discontinuous objective function (e.g. regions with invalid solutions)

The following are the common approaches in dealing with non-differential objective functions:

Non-gradient Based Methods