f = x⁴ from 1
x₁ = 1 − f′/f″ = 1 − 4/12 = 0.6667
Newton on f′.
Newton's method for minimisation updates , where is the Hessian. In one variable: . It is Newton–Raphson applied to .
This page is easier if you are comfortable with:
For a quadratic, one step lands on the minimum: , from : , , . In two variables, has Hessian , and from the gradient is , so (the Hessian). For (minimum at ): , , , ; then , , , approaching 0.6931.
Near a minimum the convergence is quadratic (Newton's quadratic convergence, Newton–Raphson), far faster than gradient descent (gradient descent), but each step needs the Hessian and a linear solve, costing (Gaussian elimination cost). It may head to a maximum or saddle if the Hessian is not positive definite, so safeguards are used.
Quasi-Newton methods (BFGS) approximate cheaply. For convex functions (convex functions) the Newton step is a descent direction; with constraints, see the KKT conditions. The stationary point conditions are in stationary points.
Newton's method for optimisation uses the curvature to jump straight towards a minimum.
Newton's iterates 0, 1, 0.7358 close in on the minimum.
These examples are worked by hand, step by step, so you can compare your own working.
Find for from .
Find from .
Find for from .
A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.
x₁ = 1 − f′/f″ = 1 − 4/12 = 0.6667
Newton on f′.
x₂ = 0.4444
Ratio 2/3 each step.
x₁ = 1, x₂ = 0.7358
Minimum at ln 2 = 0.6931.
f = x² − 4x + 5 from any x₀ reaches 2 in one step
Exact for quadratics.
Newton’s method for minimising f applies the root-finding iteration to f′: xₙ₊₁ = xₙ − f′(xₙ)/f″(xₙ). It converges quadratically near a minimum with f″ > 0 and is exact in one step for quadratics.
Apply one step from x₀ = 1.
Minimise x² − 4x + 5 from x₀ = 0 in one step.
Take two steps from x₀ = 0. Give x₂ to 4 decimal places.
Where is the minimum of eˣ − 2x?
x + f′/f″ ✗
x − f′/f″ ✓
Step opposes the gradient.
f/f′ ✗
f′/f″ ✓
Optimising uses the derivative.
finds a maximum ✗
check f″ > 0 ✓
Newton finds stationary points.
f″ = 0 ✗
step undefined ✓
Use a different method.
Try each one on paper, then open the answer. Use the solver above to see the full working.
0
1.3333
1
3
48
32
Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.
Solve: …
Newton’s root-finding applied to f′.
x − f′(x)/f″(x).
Quadratic near a minimum.
Yes, in one step.
gradient descent.
convex functions.
More in Numerical methods and optimisation and the formula sheets.