Best low-rank Approximation Explained with Worked Examples

The best rank- approximation to a matrix keeps its largest singular values and discards the rest. The error is in the spectral norm and in the Frobenius norm (the Eckart–Young theorem).

Write 2x or 2*x to multiply, ^ for powers, and ; between simultaneous equations. You can also type factor, expand, derivative of, integrate … from a to b, limit … as x->0 or 15% of 80.

Try an example

Before you start

This page is easier if you are comfortable with:

What Best low-rank approximation (Eckart–Young) means

Write the SVD as a sum of rank-one pieces . Keeping only the biggest terms captures most of the matrix with far less data, which is the idea behind image compression and noise reduction.

The error is read straight from the discarded singular values using the spectral and Frobenius norms. If a matrix has rank , its best rank- approximation is itself and the error is zero.

Compare with fitting a line to data in least squares: both find the closest simple object to a complicated one, with closeness measured by sums of squares. The sizes of the singular values are explained in singular values and norms, the idea of "closest" is the one behind least squares, and the rank itself is described by the rank-nullity theorem.

What is it used for?

Keeping only the largest singular values gives the closest matrix of lower rank, which is how data is compressed.

How to use best low-rank approximation (eckart–young)

  1. Compute the SVD and list the singular values.
  2. Keep the first terms of the sum.
  3. The spectral error is .
  4. The Frobenius error is the root of the sum of the squares of the discarded values.

Graph

Singular values 5, 3, 1: keeping two leaves an error of 1σ₁ = 5σ₂ = 3σ₃ = 1 (error)00.511.522.533.540123456Singular values 5, 3, 1: keeping two leaves an error of 1xyGraph with x from 0 to 4 and y from 0 to 6. Shows bars (3); point σ₁ = 5 at (1, 5); point σ₂ = 3 at (2, 3); point σ₃ = 1 (error) at (3, 1).

The error of the best rank-2 approximation equals the first singular value that is dropped.

Worked examples

These examples are worked by hand, step by step, so you can compare your own working.

Example 1: a diagonal matrix

Find the best rank-2 approximation of and its spectral error.

Worked example
Answer
diag⁡(3,2,0), error 1\operatorname{diag}(3,2,0),\ \text{error }1
  1. Keep σ = 3 and 2
  2. Error

Example 2: Frobenius error

For singular values 4, 3, 2, 1, find the Frobenius error of the best rank-2 approximation.

Worked example
Answer
5≈2.236\sqrt5\approx2.236
  1. Discarded values 2 and 1

Example 3: a rank-one approximation

Find the best rank-1 approximation to .

Worked example
Answer
(3000)\begin{pmatrix}3&0\\0&0\end{pmatrix}
  1. Keep σ = 3
  2. Error

Special cases

A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.

σ = (5, 3, 1), rank 1

spectral error σ₂

3.

same

Frobenius error √(9 + 1)

3.1623.

rank 2

spectral error σ₃

1.

rank 1

energy kept 25/35

0.7143.

Word problems

Keep the k largest singular values. The error is σ_{k+1} (spectral) and √(σ²_{k+1} + σ²_{k+2} + …) (Frobenius) — the best possible for rank k.

Spectral error

Rank 1: next singular value.

  1. σ₂
  2. Check: ✓3

Frobenius error

√(3² + 1²).

  1. √10
  2. Check: ✓3.1623

Rank 2 Higher

Discarded σ₃.

  1. 1
  2. Check: ✓1

Energy Higher

Σσ² = 35; kept 25.

  1. 25/35
  2. Check: ✓0.7143

Common mistakes

Keep the smallest σ

smallest ✗

largest ✓

Eckart–Young.

Error σ_k

σ_k ✗

σ_{k+1} ✓

Next discarded.

Frobenius error is σ_{k+1}

only one ✗

root of the sum of the discarded squares ✓

All discarded.

Any rank-k matrix is as good

any ✗

SVD truncation is optimal ✓

Optimality.

Exam tips

  1. Truncate the SVD.
  2. Spectral error σ_{k+1}.
  3. Frobenius error √Σσ².
  4. Energy = Σσ².
  5. Eckart–Young guarantees optimality.

Practice questions

Try each one on paper, then open the answer. Use the solver above to see the full working.

1. Spectral error: σ = (6, 4, 2), rank 1 Foundation

4

2. Frobenius error: σ = (6, 4, 2), rank 1, 4 dp Foundation

4.4721

3. Spectral error: σ = (6, 4, 2), rank 2 Foundation

2

4. Energy kept: σ = (3, 4), rank 1 Higher

0.64

5. Frobenius error: σ = (5, 12, 0), rank 1 Higher

12

6. Frobenius error: σ = (3, 2, 2, 1), rank 2, 4 dp Higher

2.2361

Practice generator

Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.

Solve: …

Key terms

Low-rank approximation
Keep k components.
Truncated SVD
Drop small σ.
Eckart–Young theorem
Optimality result.
Spectral error
σ_{k+1}.
Frobenius error
√(Σ discarded σ²).
Energy
Σσ².

Frequently asked questions

How do I get the best rank-k approximation?

Keep the k largest singular values in the SVD.

What is the spectral-norm error?

σ_{k+1}.

What is the Frobenius error?

√(σ²_{k+1} + σ²_{k+2} + …).

Why is it optimal?

By the Eckart–Young theorem.

Which page covers the SVD?

singular value decomposition.

Which page covers norms?

singular values and norms.

More in Vectors and matrices and the formula sheets.