Surrogate Objective Methods

“知行合一. – 王阳明”

Use a surrogate function to approximate the objective, take optimization steps based on the approximation, improve the approximation through experiences gained from the step, and take the next optimization step based on improved approximation.

• Quadratic: uses a linear combination of second-order and first-order terms of the inputs, which represents a convex or concave bowl shape in 3D space.

• Kriging: A fancy way of saying data interpolation.

• Radial basis function: transforming the inputs into a high-dimensional feature space, and then modeling the relationship between the inputs and outputs using a set of radial basis functions. The radial basis functions are shaped like “humps” and can be positioned anywhere in the feature space, allowing the RBF model to approximate any non-linear relationship.

• Artificial Neural Network: A network with layers of interconnected nodes, each with an adjustable weight and biase, allowing it to learn complex non-linear relationships through training on data.