Finite difference¶
Numerical approximation for the derivative of a numerical function
Why¶
- not always possible to analytically differentiate a function
- sometimes the "function" is a complicated procedure
Method¶
Use Taylor expansion:
$$f(x_0+\Delta x) \approx f(x_0) + f'(x_0)\Delta x$$Solve for the derivative:
$$f'(x_0) \approx \frac{f(x_0+\Delta x) - f(x_0)}{\Delta x}$$I need 2 evaluations of the function $f$ to estimate the derivative. For the derivative at location $x$ I have three options:
- forward difference (use $x$ and $x+\Delta x$)
- backward difference (use $x-\Delta x$ and $x$)
- central difference (use $x-\Delta x$ and $x+\Delta x$)
Which one is best depends on the situation:
- interval where $f$ is defined (evaluating $f(x_0+\Delta x)$ or $f(x_0-\Delta x)$ might not always be possible)
- if evaluation of $f$ is expensive and we are interested in the value at $x_0$ forward or backward difference might be better
Error estimate for forward difference¶
Use Taylor expansion again: $$\frac{f(x_0+\Delta x) - f(x_0)}{\Delta x} = \frac{f(x_0)+\Delta x f'(x_0)+\frac12\Delta x^2f''(x_0)+\mathcal{O}(\Delta x^3) - f(x_0)}{\Delta x} $$
$$= f'(x_0)+\frac12 f''(x_0)\Delta x +\mathcal{O}(\Delta x^2)$$The error is (approximately) proportional to $\Delta x$
Error estimate for backward difference¶
Similarily for the backward difference: $$\frac{f(x_0) - f(x_0-\Delta x)}{\Delta x} = \frac{f(x_0)- \left( f(x_0) -\Delta x f'(x_0)+\frac12\Delta x^2f(x_0)+\mathcal{O}(\Delta x^3) \right)}{\Delta x} $$ $$= f'(x_0)-\frac12 f''(x_0)\Delta x +\mathcal{O}(\Delta x^2)$$
The error is (approximately) also proportional to $\Delta x$
Error estimate for central difference¶
The central difference is better: $$\frac{f(x_0+\Delta x) - f(x_0-\Delta x)}{2\Delta x} = \frac{\left(f(x_0) +\Delta x f'(x_0)+\frac12\Delta x^2f''(x_0)+\mathcal{O}(\Delta x^3)\right) - \left( f(x_0) -\Delta x f'(x_0)+\frac12\Delta x^2f''(x_0)+\mathcal{O}(\Delta x^3) \right)}{2\Delta x} $$ $$ = f'(x_0)+\mathcal{O}(\Delta x^2)$$ The error has a higher polynomial dependency in the small parameter $\Delta x$.
Assignment 1¶
We have covered now the material for the coding in the first assignment.
It's worth having a go at it now to help you understand the material in the lectures with a real example.