Contents

• 1  Exercise
  • 1.1  Approximate derivative with the forward formula
  • 1.2  Approximate derivative with the backward formula
  • 1.3  Approximate derivative with the centered formula
  • 1.4  Compute the error
  • 1.5  Order of the forward formula
  • 1.6  Order of the backward formula
  • 1.7  Order of the centered formula

Exercise

Let $f(x)=e^{x}$

1. Compute its numerical derivative in $x_{0}=0$ with $h=1$ and $h=0.1$ using the centered, forward and backward formulas.
2. Compute the absolute error in each case.
3. Study the order of each formula.

---

The definition of derivative of $f$ at the point $x_0$ is

$$ f^{\prime}(x_0)=\lim_{x\to x_0}\frac{f(x)-f(x_0)}{x-x_0} $$

We can obtain from it the three approximations of the derivative or finite difference formulas

Approximate derivative with the forward formula

If $x = x_0+h$ with $h \gt 0$ then $h = x-x_0$ and

$$ f^{\prime}(x_0)\approx\frac{f(x_0+h)-f(x_0)}{h}$$

For $x_0=0$ and $h=1$ $$f^{\prime}(0)\approx\frac{f(0+1)-f(0)}{1}=\frac{2.72-1}{1}=1.72$$

And for $x_0=0$ with $h=0.1$ $$f^{\prime}(0)\approx\frac{f(0+0.1)-f(0)}{h}=\frac{1.105-1}{0.1}=1.05$$

For these formulas, geometrically

• The exact derivative of a function $f$ at a point $x_0$ gives us the slope of the tangent line to the curve at the point.
• The approximate derivative of a function $f$ at a point $x_0$ gives us the slope of the line that is a secant to the curve at the two points that we use in the formula.

Therefore, the more similar these two lines, the more similar the exact value and the approximation. In the graph, for example, they are clearly different because $h$ is very large. If the calculations would not have an error (with finite arithmetic they do), the smaller the $h$, the smaller the error of the approximation.

Approximate derivative with the backward formula

If $x = x_0-h$ wit $h \gt 0$ then $-h = x-x_0$ and

$$f^{\prime}(x_0)\approx\frac{f(x_0 -h)-f(x_0)}{-h}=\frac{f(x_0)-f(x_0 -h)}{h}$$

For $x_0=0$ and $h = 1$ $$f^{\prime}(0)\approx\frac{f(0)-f(0-1)}{1}=\frac{1-0.37}{1}=0.63$$

For $x_0=0$ and $h = 0.1$ $$f^{\prime}(0)\approx\frac{f(0)-f(0-0.1)}{0.1}=\frac{1-0.905}{0.1}=0.95$$

Approximate derivative with the centered formula

If we average the forward and backward formulas

$$f^{\prime}(x_0)\approx \frac{1}{2}\left(\frac{f(x_0+h)-f(x_0)}{h}+\frac{f(x_0)-f(x_0 -h)}{h}\right)= \frac{f(x_0+h)-f(x_0-h)}{2h}$$

For $x_0=0$ and $h = 1$ $$f^{\prime}(0)\approx \frac{f(0+1)-f(0-1)}{2(1)}=\frac{2.72-0.37}{2(1)}=1.18$$

For $x_0=0$ and $h = 0.1$ $$f^{\prime}(0)\approx \frac{f(0+0.1)-f(0-0.1)}{2(0.1)}=\frac{1.1052- 0.9048}{2(0.1)}=1.002$$

In this case, the slopes of the tangent line and the secant line, despite being $h$ quite large, are similar. So it is expected that the exact and approximate values ​​will be more similar than in the previous cases. This is because this is a formula of order two, while the previous two formulas are of order one.

Compute the error

If $y$ is the actual value and $y^*$ the approximation:

• Absolute error $$e_a=|y-y^*|$$
• Relative error $$e_r=\frac{e_a}{|y|}$$

• For $h = 1$
  • Forward formula
    • Absolute error $$e_a=|y-y^*|= |1.00-1.72|=0.72$$
    • Relative error $$e_r=\frac{e_a}{|y|}=\frac{0.72}{1}=0.72=72\%$$
  • Backward formula
    • Absolute error $$e_a=|y-y^*|= |1.00-0.63|=0.37$$
    • Relative error $$e_r=\frac{e_a}{|y|}=\frac{0.37}{1}=0.37=37\%$$
  • Centered formula
    • Absolute error $$e_a=|y-y^*|= |1.00-1.18|=0.18$$
    • Relative error $$e_r=\frac{e_a}{|y|}=\frac{0.18}{1}=0.18=18\%$$

• For $h = 0.1$
  • Forward formula
    • Absolute error $$e_a=|y-y^*|= |1.00-1.05|=0.05$$
    • Relative error $$e_r=\frac{e_a}{|y|}=\frac{0.05}{1}=0.05=5\%$$
  • Backward formula
    • Absolute error $$e_a=|y-y^*|= |1.00-0.95|=0.05$$
    • Relative error $$e_r=\frac{e_a}{|y|}=\frac{0.37}{1}=0.05=5\%$$
  • Centered formula
    • Absolute error $$e_a=|y-y^*|= |1.000-1.002|=0.002$$
    • Relative error $$e_r=\frac{e_a}{|y|}=\frac{0.002}{1}=0.002=0.2\%$$

We see that for the same $h$ the centered formula gives a smaller error. And also that the smaller the $h$, the smaller the error (up to a certain point, because if $h$ is too small, rounding errors dominate and the error increases again).

Order of the forward formula

We say that the order of a numerical differentiation formula is $n$ if the absolute error for this formula is

$$E_h = K\,h^n$$

Since $h$ is a small value, the larger $n$ the smaller the error in the formula (if $h=0.1$ then $h^2 = 0.01$ and $h^3 = 0.001$)

The forward formula is

$$ f'(a)\approx \frac{f(a+h)-f(a)}{h} $$

Taylor's series formula is

$$f(x)=f(x_0)+f'(x_0)\frac{x-x_0}{1!}+f''(c)\frac{(x-x_0)^2}{2!} \quad c \in(x,x_0)\;\mathrm{or}\; c\in (x_0,x)$$

If $x_0=a$ and $x=a+h$ then $h = x-x_0$ and

$$f(a+h)=f(a)+f'(a)\frac{h}{1!}+f''(c)\frac{h^2}{2!} \quad c \in(a+h,a) \;\mathrm{or}\; c\in (a,a+h)$$

If we take into account the progressive formula, subtracting $f(a)$ from both sides

$$f(a+h)-f(a)=f'(a)\frac{h}{1!}+f''(c)\frac{h^2}{2!}$$

and dividing by $h$

$$\frac{f(a+h)-f(a)}{h}=f'(a)+f''(c)\frac{h}{2}$$

also

$$-f''(c)\frac{h}{2}=f'(a)-\frac{f(a+h)-f(a)}{h}$$

And since the error is the exact value minus the approximate

$$E_h =f'(a)-\frac{f(a+h)-f(a)}{h}$$

and the error is

$$E_h =-f''(c)\frac{h}{2}= K\,h$$

and since the exponent of the $h$ gives us the order of the formula, this is a formula of order 1.

Order of the backward formula

$$ f'(a)\approx \frac{f(a)-f(a-h)}{h} $$

Taylor's series formula is

$$f(x)=f(x_0)+f'(x_0)\frac{x-x_0}{1!}+f''(c)\frac{(x-x_0)^2}{2!} \quad c \in(x,x_0)\;\mathrm{or}\; c\in (x_0,x)$$

If $x_0=a$ and $x=a-h$ then $-h = x-x_0$ and

$$f(a-h)=f(a)+f'(a)\frac{-h}{1!}+f''(c)\frac{(-h)^2}{2!} \quad c \in(a+h,a) \;\mathrm{or}\; c\in (a,a+h)$$

If we take into account the forward formula, subtracting $f(a)$ from both sides

$$f(a-h)-f(a)=-f'(a)\frac{h}{1!}+f''(c)\frac{h^2}{2!}$$

and dividing by $-h$

$$\frac{f(a-h)-f(a)}{-h}=f'(a)-f''(c)\frac{h}{2}$$

that is

$$\frac{f(a)-f(a-h)}{h}=f'(a)-f''(c)\frac{h}{2}$$

also

$$f''(c)\frac{h}{2}=f'(a)-\frac{f(a)-f(a-h)}{h}$$

and since the error is the exact value minus the approximate

$$E_h =f'(a)-\frac{f(a)-f(a-h)}{h}$$

the error is

$$E_h =f''(c)\frac{h}{2}= K\,h$$

and since the exponent of the $h$ gives us the order of the formula, this is a formula of order 1.

Order of the centered formula

$$ f'(a)\approx \frac{f(a+h)-f(a-h)}{2h} $$

Taylor's series formula is

$$f(x)=f(x_0)+f'(x_0)\frac{x-x_0}{1!}+f''(x_0)\frac{(x-x_0)^2}{2!} +f'''(c)\frac{(x-x_0)^3}{3!}$$

with

$$ c \in(x,x_0)\;\mathrm{or}\; c\in (x_0,x) $$

If $x_0=a$ and $x=a+h$ then $h = x-x_0$ and

$$f(a+h)=f(a)+f'(a)\frac{h}{1!}+f''(a)\frac{h^2}{2!} +f'''(c_1)\frac{h^3}{3!}$$

If $x_0=a$ and $x=a-h$ then $-h = x-x_0$ and

$$f(a-h)=f(a)+f'(a)\frac{-h}{1!}+f''(a)\frac{(-h)^2}{2!} +f'''(c_2)\frac{(-h)^3}{3!} $$

Y $-f(a-h)$ is

$$-f(a-h)=-f(a)+f'(a)\frac{h}{1!}-f''(a)\frac{h^2}{2!} +f'''(c_2)\frac{h^3}{3!} $$

If we take into account the centered formula, adding $f(a+h)+(-f(a-h))$ to both sides and taking into account that if $f'''$ is continuous, because of the intermediate value theorem, we can find a value $c_3$ in $(c_1,c_2)$ such that $$f'''(c_3)=\frac{f'''(c_1)+f'''(c_2)}{2}$$ then

$$f(a+h)-f(a-h)=2f'(a)h+2f'''(c_3)\frac{h^3}{6}$$

and dividing by $2h$

$$\frac{f(a+h)-f(a-h)}{2h}=f'(a)+f'''(c_3)\frac{h^2}{6}$$

also

$$-f'''(c_3)\frac{h^2}{6}=f'(a)-\frac{f(a+h)-f(a-h)}{2h}$$

and since the error is the exact value minus the approximate

$$E_h =f'(a)-\frac{f(a+h)-f(a-h)}{2h}$$

the error is

$$E_h =-f'''(c)\frac{h^2}{6}= K\,h^2$$

and since the exponent of the $h$ gives us the order of the formula, this is a formula of order 2.