Contents

• 1  Analytically: with Lagrange multipliers
• 2  Numerically: penalty function

Local minimum with equality constraints. Symbolic and numerical calculus

Minimize the function $f$ with the constraint $c$ where $$f(x,y)=-y \qquad c: x^{2}+y^{2}=1$$

---

Analytically: with Lagrange multipliers

Interactive plot

We build the Lagrangian function $$ L(x,y,\lambda)= f(x,y) + \lambda\,g(x,y) $$

that is

$$ L(x,y,\lambda)=-y + \lambda\,(x^2+y^2-1) $$

We compute the gradient $L$ and make it zero

$$ \begin{align} L^{\prime}_{x}&=&2x\lambda&=&0 \\ L^{\prime}_{y}&=&-1+2y\lambda&=&0 \\ L^{\prime}_{\lambda}&=&x^2+y^2-1&=&0 \end{align} $$

As $\lambda \ne 0$ from the first equation we get $x=0$. Substituting this value of $x$ in the third equation we have $y = \pm 1$ and substituting these values in the second equation we obtain the corresponding values of $\lambda.$ We have two solutions

$$ x = 0 \quad y = -1 \quad \lambda = -\frac{1}{2} \qquad \mathrm{and} \qquad x = 0 \quad y = 1 \quad \lambda = \frac{1}{2} $$

To check whether this point is a maximum or a minimum we will use the determinant of the Hessian matrix of the Lagrangian $L(x,y,\lambda)$ calculated at the critical point.

• If negative, the point is a minimum.
• If positive, the point is a maximum.

The Hessian matrix of the Lagrangian is

$$ H_{L}= \left( \begin{array}{ccc} L^{\prime\prime}_{xx} & L^{\prime\prime}_{xy} & L^{\prime\prime}_{x\lambda}\\ L^{\prime\prime}_{yx} & L^{\prime\prime}_{yy} & L^{\prime\prime}_{y\lambda}\\ L^{\prime\prime}_{\lambda x} & L^{\prime\prime}_{\lambda y} & L^{\prime\prime}_{\lambda\lambda}\\ \end{array} \right) $$

that is

$$ H_{L} = \left( \begin{array}{rrc} 2\lambda & 0 & 2x\\ 0 & 2\lambda & 2y\\ 2x & 2y & 0\\ \end{array} \right). $$

For

$$ x = 0 \quad y = -1 \quad \lambda = -\frac{1}{2} \qquad H_{L} = \left(\begin{array}{rrr} -1 & 0 & 0\\ 0 & -1 & -2\\ 0 & -2 & 0\\ \end{array}\right) \qquad \det(H_{L})=4 \gt 0 $$

and it is a maximum.

For

$$ x = 0 \quad y = 1 \quad \lambda = \frac{1}{2} \qquad H_{L} = \left(\begin{array}{rrc} 1 & 0 & 0\\ 0 & 1 & 2\\ 0 & 2 & 0\\ \end{array}\right) \qquad \det(H_{L})=-4 \lt 0 $$

and it is a minimum that we were searching for.

Numerically: penalty function

The idea of the penalty method is to replace the objective function $f$ by another function

$$F(x,y)=f(x,y)+c\,P(x,y)$$

and solve the problem without restrictions. For this, we take $c$ as a positive constant and we must choose $P$, the penalty function, in such a way that:

• $P$ is continuous in the domain of $f$.
• $P(x,y)\geq 0$ for every point in the domain of $f$, and
• $P(x,y)=0$ if and only if the point $(x,y)$ satisfies the constraints.

A possible function to approximate the constrained minimum would be

$$ F(x,y) = -y + 10\,(x^2+y^2-1)^2$$