σ = (5, 3, 1), rank 1
spectral error σ₂
3.
The best rank- approximation to a matrix keeps its largest singular values and discards the rest. The error is in the spectral norm and in the Frobenius norm (the Eckart–Young theorem).
This page is easier if you are comfortable with:
Write the SVD as a sum of rank-one pieces . Keeping only the biggest terms captures most of the matrix with far less data, which is the idea behind image compression and noise reduction.
The error is read straight from the discarded singular values using the spectral and Frobenius norms. If a matrix has rank , its best rank- approximation is itself and the error is zero.
Compare with fitting a line to data in least squares: both find the closest simple object to a complicated one, with closeness measured by sums of squares. The sizes of the singular values are explained in singular values and norms, the idea of "closest" is the one behind least squares, and the rank itself is described by the rank-nullity theorem.
Keeping only the largest singular values gives the closest matrix of lower rank, which is how data is compressed.
The error of the best rank-2 approximation equals the first singular value that is dropped.
These examples are worked by hand, step by step, so you can compare your own working.
Find the best rank-2 approximation of and its spectral error.
For singular values 4, 3, 2, 1, find the Frobenius error of the best rank-2 approximation.
Find the best rank-1 approximation to .
A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.
spectral error σ₂
3.
Frobenius error √(9 + 1)
3.1623.
spectral error σ₃
1.
energy kept 25/35
0.7143.
Keep the k largest singular values. The error is σ_{k+1} (spectral) and √(σ²_{k+1} + σ²_{k+2} + …) (Frobenius) — the best possible for rank k.
Rank 1: next singular value.
√(3² + 1²).
Discarded σ₃.
Σσ² = 35; kept 25.
smallest ✗
largest ✓
Eckart–Young.
σ_k ✗
σ_{k+1} ✓
Next discarded.
only one ✗
root of the sum of the discarded squares ✓
All discarded.
any ✗
SVD truncation is optimal ✓
Optimality.
Try each one on paper, then open the answer. Use the solver above to see the full working.
4
4.4721
2
0.64
12
2.2361
Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.
Solve: …
Keep the k largest singular values in the SVD.
σ_{k+1}.
√(σ²_{k+1} + σ²_{k+2} + …).
By the Eckart–Young theorem.
singular value decomposition.
singular values and norms.
More in Vectors and matrices and the formula sheets.