Contents

• 1  Exercise
  • 1.1  Gradient $\nabla f(x,y)$
  • 1.2  Hessian matrix $\mathrm{Hess} f(x,y)$

Exercise

Given $f(x,y)=x^{3}+y^{3}$, compute an approximation of

1. The gradient $\nabla f(x,y)$
2. The Hessian matrix $\mathrm{Hess} f(x,y)$

for $(x_0,y_0)=(1,2)$ with $h_{x}=h_{y}=0.1$.

Gradient $\nabla f(x,y)$

The gradient is $$ \nabla f=\left(\frac{\partial f}{\partial x},\frac{\partial f}{\partial y}\right) $$

We compute the exact gradient calculating the partial derivatives of $f$

$$\frac{\partial f}{\partial x} = 3x^2 \quad \frac{\partial f}{\partial y}=3y^2$$

that for the point $(x_0,y_0)=(1,2)$

$$\left(\frac{\partial f}{\partial x}\right)_{(1,2)} = 3(1)^2=3 \quad \left(\frac{\partial f}{\partial y}\right)_{(1,2)}=3(2)^2=12$$

And finally

$$\nabla f(x,y)_{(1,2)}=(3,12).$$

Let us compute an approximation. The first component of the gradient would be

$$ \begin{align*} \frac{\partial f}{\partial x}\approx & \dfrac{f\left(x_{m}+h_{x},y_{n}\right)-f\left(x_{m}-h_{x},y_{n}\right)}{2h_{x}}=\dfrac{f\left(1+0.1,2\right)-f\left(1-0.1,2\right)}{2\left(0.1\right)}=\\[0.3cm] = & \dfrac{\left(\left(1+0.1\right)^{3}+\left(2\right)^{3}\right)-\left(\left(1-0.1\right)^{3}+\left(2\right)^{3}\right)}{2\left(0.1\right)}=\\[0.3cm] = & \dfrac{9.331-8.729}{0.2}=\dfrac{0.602}{0.2}=3.01\\ \end{align*} $$

And the second component

$$ \begin{align*} \frac{\partial f}{\partial y}\approx & \dfrac{f\left(x_{m},y_{n}+h_{y}\right)- f\left(x_{m},y_{n}-h_{y}\right)}{2h_{y}}=\dfrac{f\left(1,2+0.1\right)-f\left(1,2-0.1\right)}{2\left(0.1\right)}=\\[0.3cm] = & \dfrac{\left(\left(1\right)^{3}+\left(2+0.1\right)^{3}\right)-\left(\left(1\right)^{3}+\left(2-0.1\right)^{3}\right)}{2\left(0.1\right)}=\\[0.3cm] = & \dfrac{10.261-7.859}{0.2}=\dfrac{2.401}{0.2}=12.01\\ \end{align*} $$

Thus

$$ \nabla f(1,2)\approx (3.01,12.01)$$

Hessian Matrix $\mathrm{Hess} f(x,y)$

The exact Hessian matrix of a two variable function is $$ H = \left( \begin{array}{cc} \dfrac{\partial^{2}f}{\partial x^{2}} & \dfrac{\partial^{2}f}{\partial x\partial y}\\ \dfrac{\partial^{2}f}{\partial y\partial x}& \dfrac{\partial^{2}f}{\partial y^{2}} \end{array} \right) $$

As

$$\frac{\partial f}{\partial x} = 3x^2 \quad \frac{\partial f}{\partial y}=3y^2$$

And the second partial derivatives are

$$ \frac{\partial^{2}f}{\partial x^{2}} = 6x \quad \frac{\partial^{2}f}{\partial y^{2}}=6y $$$$ \dfrac{\partial^{2}f}{\partial x\partial y} = 0 \quad \dfrac{\partial^{2}f}{\partial y\partial x} = 0 $$

That for the point $(x_0,y_0)=(1,2)$

$$ \left(\frac{\partial^{2}f}{\partial x^{2}}\right)_{(1,2)} = 6(1)=6 \quad \left(\frac{\partial^{2}f}{\partial y^{2}}\right)_{(1,2)}=6(2)=12 \quad \left(\frac{\partial^{2}f}{\partial x\partial y}\right)_{(1,2)}=\left(\frac{\partial^{2}f}{\partial y\partial x}\right)_{(1,2)}=0 $$

And finally

$$ H = \left( \begin{array}{cc} 6 & 0 \\ 0 & 12 \end{array} \right) $$

Let us compute an approximation. Using the second order partial derivative formula we already have

$$ \begin{align*} \frac{\partial^{2}f}{\partial x^{2}}& \approx \dfrac{f\left(x_{m}+h_{x},y_{n}\right)-2f\left(x_{m},y_{n}\right)+f\left(x_{m}-h_{x},y_{n}\right)}{h_{x}^{2}}=\\[0.3cm] &= \dfrac{f\left(1+0.1,2\right)-2f\left(1,2\right)+f\left(1-0.1,2\right)}{\left(0.1\right)^{2}}=\\[0.3cm] & = \dfrac{\left(\left(1+0.1\right)^{3}+\left(2\right)^{3}\right)-2\left(\left(1\right)^{3}+\left(2\right)^{3}\right)+\left(\left(1-0.1\right)^{3}+\left(2\right)^{3}\right)}{\left(0.1\right)^{3}}=\\[0.3cm] &= \dfrac{9.331-18+8.729}{0.01}=\dfrac{0.06}{0.01}=6\\ \end{align*} $$

And $$ \begin{align*} \frac{\partial^{2}f}{\partial y^{2}} & \approx\dfrac{f\left(x_{m},y_{n}+h_{y}\right)-2f\left(x_{m},y_{n}\right)+f\left(x_{m},y_{n}-h_{y}\right)} {h_{y}^{2}}=\\[0.3cm] &=\dfrac{f\left(1,2+0.1\right)-2f\left(1,2\right)+f\left(1,2-0.1\right)}{\left(0.1\right)^{2}}=\\[0.3cm] & =\dfrac{\left(\left(1\right)^{3}+\left(2+0.1\right)^{3}\right)-2\left(\left(1\right)^{3}+\left(2\right)^{3}\right)+\left(\left(1\right)^{3}+\left(2-0.1\right)^{3}\right)}{\left(0.1\right)^{2}}=\\[0.3cm] & = \dfrac{10.261-18+7.859}{0.01}=\dfrac{0.12}{0.01}=12\\ \end{align*} $$

Let us compute the cross partial derivatives. We will calculate only one because, under certain conditions specified by Schwarz Theorem (met in this case), the partial cross derivatives are equal. Let us use centered formulas

$$ \frac{\partial^{2}f}{\partial y\partial x}=\frac{\partial}{\partial y}\left(\frac{\partial f}{\partial x}\right) = \frac{\partial f'_x}{\partial y} \approx \dfrac{f'_x\left(x_{m},y_{n}+h_{y}\right)-f'_x\left(x_{m},y_{n}-h_{y}\right)} {2h_{y}} $$

And taking into account

$$ f'_x\left(x_{m},y_{n}+h_{y}\right) \approx \dfrac{f\left(x_{m}+h_{x},y_{n}+h_{y}\right)-f\left(x_{m}-h_{x},y_{n}+h_{y}\right)}{2h_{x}} $$

and

$$ f'_x\left(x_{m},y_{n}-h_{y}\right) \approx \dfrac{f\left(x_{m}+h_{x},y_{n}-h_{y}\right)-f\left(x_{m}-h_{x},y_{n}-h_{y}\right)}{2h_{x}} $$

we have

$$ \frac{\partial^{2}f}{\partial y\partial x}\approx \frac{f\left(x_{m}+h_{x},y_{n}+h_{y}\right)-f\left(x_{m}-h_{x},y_{n}+h_{y}\right)-f\left(x_{m}+h_{x},y_{n}-h_{y}\right)+f\left(x_{m}-h_{x},y_{n}-h_{y}\right)}{4\,h_x\,h_y} $$

For the given point

$$ \dfrac{f(1+0.1,2+0.1)-f(1-0.1,2+0.1)-f(1+0.1,2-0.1)+f(1-0.1,2-0.1)}{4(0.1)(0.1)}= $$

that is

$$ =\dfrac{(1.1^3+2.1^3)-(0.9^3+2.1^3)-(1.1^3+1.9^3)+(0.9^3+1.9^3)}{4(0.1)^2}= $$

And using three decimal digits

$$ =\dfrac{10.592-9.990-8.190+7.588}{0.04}=\dfrac{0}{0.04}=0 $$

And the approximate Hessian matrix is

$$ H \approx \left( \begin{array}{cc} 6 & 0 \\ 0 & 12 \end{array} \right) $$