points (1,2), (2,3), (3,5)
slope b = (nΣxy − ΣxΣy)/(nΣx² − (Σx)²)
1.5.
The least-squares solution of minimises and satisfies the normal equations . It gives the line or curve of best fit through data.
This page is easier if you are comfortable with:
When there are more equations than unknowns, no exact solution may exist. The best approximation makes the error vector perpendicular to every column of , which is the orthogonal projection of onto the column space. That condition is the normal equations.
For a straight line the equations reduce to the familiar formulas for the regression line. Solving the normal equations is an application of solving a system with an inverse.
When the columns are dependent, use the pseudoinverse, and for numerical stability the QR factorisation.
Least squares finds the line or curve that comes closest to a set of points, measured by the sum of squared vertical errors.
The line is not through every point; it makes the total of the squared gaps as small as possible.
These examples are worked by hand, step by step, so you can compare your own working.
Fit a line to , , .
Fit a line to , , , .
For the first fit, show the residuals sum to zero.
A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.
slope b = (nΣxy − ΣxΣy)/(nΣx² − (Σx)²)
1.5.
intercept a = (Σy − bΣx)/n
0.3333.
a + 2b
3.3333.
5 − (a + 3b)
0.1667.
The least-squares solution of Ax ≈ b minimises |Ax − b|² and satisfies the normal equations AᵀAx = Aᵀb. For a line y = a + bx it gives the usual regression formulas.
n = 3, Σx = 6, Σy = 10, Σx² = 14, Σxy = 23.
(10 − 9)/3.
a + 2b.
y − fitted.
absolute ✗
sum of squares ✓
Least squares.
exact ✗
normal equations ✓
Overdetermined.
positive ✗
they sum to 0 with an intercept ✓
Property.
fine ✗
columns must be independent ✓
Else non-unique.
Try each one on paper, then open the answer. Use the solver above to see the full working.
2
1
0.5
0.6667
2
-0.3333
Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.
Solve: …
The x minimising |Ax − b|².
AᵀAx = Aᵀb.
Ax is the projection of b onto the column space; see orthogonal projection.
When the columns of A are independent.
Moore–Penrose pseudoinverse.
QR and Cholesky factorisations.
More in Vectors and matrices and the formula sheets.