Table of Content Introduction Unconstrained Gradient-based Fixed Step Methods Searched Step Methods Second Order Methods Trust Region Methods Constrained Optimization Lagrangian Multiplier Method Sequential Quadratic Programming Interior Point Method Gradient Free Pattern Search Methods Stochastic Methods Population Based Methods Surrogate Objective Methods Multi-Objective Uncertainties