Calculus Optimization Methods/Lagrange Multipliers

The method of Lagrange multipliers solves the constrained optimization problem by transforming it into a non-constrained optimization problem of the form:

• $\operatorname {\mathcal {L}} (x_{1},x_{2},\ldots ,x_{n},\lambda )=\operatorname {f} (x_{1},x_{2},\ldots ,x_{n})+\operatorname {\lambda } (k-g(x_{1},x_{2},\ldots ,x_{n}))$

Then finding the gradient and Hessian as was done above will determine any optimum values of $\operatorname {\mathcal {L}} (x_{1},x_{2},\ldots ,x_{n},\lambda )$.

Suppose we now want to find optimum values for $f(x,y)=2x^{2}+y^{2}$ subject to $x+y=1$ from [2].

Then the Lagrangian method will result in a non-constrained function.

• $\operatorname {\mathcal {L}} (x,y,\lambda )=2x^{2}+y^{2}+\lambda (1-x-y)$

The gradient for this new function is

• ${\frac {\partial {\mathcal {L}}}{\partial x}}(x,y,\lambda )=4x+\lambda (-1)=0$
• ${\frac {\partial {\mathcal {L}}}{\partial y}}(x,y,\lambda )=2y+\lambda (-1)=0$
• ${\frac {\partial {\mathcal {L}}}{\partial \lambda }}(x,y,\lambda )=1-x-y=0$

Finding the stationary points of the above equations can be obtained from their matrix from.

${\begin{bmatrix}4&0&-1\\0&2&-1\\-1&-1&0\end{bmatrix}}{\begin{bmatrix}x\\y\\\lambda \end{bmatrix}}={\begin{bmatrix}0\\0\\-1\end{bmatrix}}$

This results in $x=1/3,y=2/3,\lambda =4/3$.

Next we can use the Hessian as before to determine the type of this stationary point.

$H({\mathcal {L}})={\begin{bmatrix}4&0&-1\\0&2&-1\\-1&-1&0\end{bmatrix}}$

Since $H({\mathcal {L}})>0$ then the solution $(1/3,2/3,4/3)$ minimizes $f(x,y)=2x^{2}+y^{2}$ subject to $x+y=1$ with $f(x,y)=2/3$.