Numerical Methods - Lecture - Newton-Raphson Method

Newton-Raphson Method

Introduction

The Newton-Raphson method is a powerful and widely-used iterative technique for finding the roots of a real-valued function $ f(x) $. Unlike the Bisection and Regula Falsi methods, which do not require the computation of derivatives, the Newton-Raphson method leverages the derivative of the function to achieve rapid convergence to the root. This method is particularly effective when an initial approximation to the root is already known.

Basic Idea

It is based on the idea of linear approximation. Starting with an initial guess $x_0$ for the root of the equation $f(x) = 0$, the method uses the tangent line to the curve of $f(x)$ at $x_0$ to generate a better approximation $x_1$. This process is repeated, producing a sequence of values $x_0, x_1, x_2, \dots,$, that converge to the actual root (look at the figure below).

We now derive an iteration scheme for the Newton's method. The equation of the tangent line passing through $(x_0, f(x_0))$ will be \[ y - f(x_0) = f'(x_0) \left( x - x_0 \right) . \] This line intersects the $x$-axis at $x_1$, which can be find by putting $y = 0$. So, \begin{align*} 0 - f(x_0) = f'(x_0) \left( x_1 - x_0 \right) & \implies x_1 - x_0 = -\frac{f(x_0)}{f'(x_0)} \\ & \implies x_1 = x_0 - \frac{f(x_0)}{f'(x_0)}. \end{align*} Note that the above computation will work only if $f'(x_0) \neq 0$. Thus, in the Newton Raphson method, we need to choose $x_0$ such that $f'(x_0)\neq 0$. Therefore, the iterative formula for the Newton-Raphson method is given by: \[ x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}, \quad n =0,1,2\dots. \]

Step 1:

Choose a point $x_0$ as the initial guess of the solution. Make sure that $f'(x_0) \neq 0$.

Step 2:

For $n = 0,1,2,\dots,$ until the error is smaller than a specified value, calculate $x_{n+1}$ by using \[ x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}. \]

Convergence

One of the main advantages of the Newton-Raphson method is its rapid convergence, especially when the initial guess is close to the actual root. Under ideal conditions, the method exhibits quadratic convergence, meaning that the number of correct digits in the approximation roughly doubles with each iteration. However, the method's performance can be hindered if the derivative $ f'(x) $ is zero or very small near the root, leading to potential convergence issues or even divergence.

Example: Find a root of the equation $x \sin x = -\cos x$ correct to four decimal places.

Solution: The given equation is \[ x \sin x= -\cos x \implies f(x) = x\sin x + \cos x. \] The derivative will be \[ f'(x) = \sin x + x \cos x - \sin x = x \cos x. \] From the Newton Raphson iteration scheme, we have \begin{align*} x_{n+1} & = x_n - \frac{f(x_n)}{f'(x_n)}, \quad n\geq 1 \\ & = x_n - \frac{x_n \sin x_n + \cos x_n}{x_n \cos x_n}, \quad n\geq 1. \end{align*} We start with an initial guess as $x_0 = \pi$.

First iteration: \begin{align*} x_1 & = x_0 - \frac{x_0 \sin x_0 + \cos x_0}{x_0 \cos x_0} \\ & = \pi - \frac{0 - 1}{-\pi } \\ & = \pi - \frac{1}{\pi} = 2.823283. \end{align*}

Second iteration: \begin{align*} x_2 & = x_1 - \frac{x_1 \sin x_1 + \cos x_1}{x_1 \cos x_1} \\ & = 2.798600. \end{align*}

Third iteration: \begin{align*} x_3 & = x_2 - \frac{x_2 \sin x_2 + \cos x_2}{x_2 \cos x_2} \\ & = 2.798386. \end{align*}

Fourth iteration: \begin{align*} x_4 & = x_3 - \frac{x_3 \sin x_3 + \cos x_3}{x_3 \cos x_3} \\ & = 2.798386. \end{align*}

The following table summarizes the computation. \[ \begin{array}{|c|c|c|c|c|} \hline n & x_{n} & f(x_n) & f'(x_n) & x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} \\ \hline 0 & \pi & -1.000000 & -\pi & 2.823283 \\ \hline 1 & 2.823283 & -0.066186 & -2.681457 & 2.798600 \\ \hline 2 & 2.798600 & -0.000564 & -2.635588 & 2.798386 \\ \hline 3 & 2.798386 & -0.000000 & -2.635185 & 2.798386 \\ \hline \end{array} \]