Numerical approximation for the derivative of a numerical function
We use Taylor expansion to approximate the function and then solve for the derivative:
For a forward difference, the Taylor expansion is:
$$f(x_0+\Delta x) \approx f(x_0) + f'(x_0)\Delta x$$
Solving for the derivative gives:
$$f'(x_0) \approx \frac{f(x_0+\Delta x) - f(x_0)}{\Delta x}$$
For a backward difference, the Taylor expansion is:
$$f(x_0-\Delta x) \approx f(x_0) - f'(x_0)\Delta x$$
Solving for the derivative gives:
$$f'(x_0) \approx \frac{f(x_0) - f(x_0-\Delta x)}{\Delta x}$$
For a central difference, we combine both the forward and backward Taylor expansions:
$$f(x_0+\Delta x) - f(x_0-\Delta x) \approx 2 f'(x_0) \Delta x$$
Solving for the derivative gives:
$$f'(x_0) \approx \frac{f(x_0+\Delta x) - f(x_0-\Delta x)}{2\Delta x}$$
We need 2 evaluations of the function \(f\) to estimate the derivative for forward and backward differences, but 3 for central difference. Which method to choose depends on the situation:
Which one is best depends on the situation:
Use Taylor expansion again: $$\frac{f(x_0+\Delta x) - f(x_0)}{\Delta x} = \frac{f(x_0)+\Delta x f’(x_0)+\frac12\Delta x^2f’’(x_0)+\mathcal{O}(\Delta x^3) - f(x_0)}{\Delta x} $$
$$= f’(x_0)+\frac12 f’’(x_0)\Delta x +\mathcal{O}(\Delta x^2)$$ The error is (approximately) proportional to $\Delta x$
Similarily for the backward difference: $$\frac{f(x_0) - f(x_0-\Delta x)}{\Delta x} = \frac{f(x_0)- \left( f(x_0) -\Delta x f’(x_0)+\frac12\Delta x^2f(x_0)+\mathcal{O}(\Delta x^3) \right)}{\Delta x} $$ $$= f’(x_0)-\frac12 f’’(x_0)\Delta x +\mathcal{O}(\Delta x^2)$$
The error is (approximately) also proportional to $\Delta x$
The central difference is better: $$\frac{f(x_0+\Delta x) - f(x_0-\Delta x)}{2\Delta x} = \frac{\left(f(x_0) +\Delta x f’(x_0)+\frac12\Delta x^2f’’(x_0)+\mathcal{O}(\Delta x^3)\right) - \left( f(x_0) -\Delta x f’(x_0)+\frac12\Delta x^2f’’(x_0)+\mathcal{O}(\Delta x^3) \right)}{2\Delta x} $$ $$ = f’(x_0)+\mathcal{O}(\Delta x^2)$$ The error has a higher polynomial dependency in the small parameter $\Delta x$.
We have covered now the material for the coding in the first assignment.
It’s worth having a go at it now to help you understand the material in the lectures with a real example.