Lagrangian Multiplier Method

“Laws without enforced consequences are merely suggestions – Ron Brackin” penalty

Summary:

we add one Lagrangian Multiplier on each constraint, and hence convert the constraints into penalty terms of objective function. Commonly used in solving optimization with equality constraints.

Intuition:

If the objective function has now included the cost of contraint violation, its stationary points will also be the new optima.

Method:

  1. Formulate the original constrained optimization problem as an unconstrained problem by introducing Lagrange multipliers as additional variables.

  2. Write the Lagrangian function, which is the sum of the original objective function and the constraint functions multiplied by their corresponding Lagrange multipliers.

  3. Find the stationary points of the Lagrangian by setting the gradient of the Lagrangian to zero and solving for the variables and multipliers.

  4. Check the second-order conditions to ensure that the stationary point is a local minimum of the Lagrangian.

  5. Substitute the optimal values of the multipliers back into the original constraints to check feasibility and optimality.