Var = 2, bias = 0.5
MSE = 2 + 0.25
2.25.
The mean squared error of an estimator is . It combines how spread out the estimator is and how far it is off on average.
This page is easier if you are comfortable with:
For a sample of from with : the unbiased estimator has variance , so . The maximum likelihood estimator has bias and variance , so : smaller, despite the bias. See the sample variance and the MLE for the normal.
A shrunk estimator of a mean 2 with has bias and variance , so MSE , against for itself: shrinking helps only when the true mean is small relative to the noise. The same trade-off is analysed in the bias–variance decomposition, and ridge regression shrinks coefficients on purpose (ridge regression).
Unbiasedness is therefore not the only criterion. For unbiased estimators the lowest possible variance is the Cramér–Rao bound (Fisher information and the Cramér–Rao bound), and conditioning on a sufficient statistic can only reduce the MSE (Rao–Blackwell).
Mean squared error rewards estimators that are both close on average and stable.
The biased MLE has the smaller error because its variance is lower.
These examples are worked by hand, step by step, so you can compare your own working.
Find the MSE of for , .
Find the MSE of for , .
Find the MSE of for , .
A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.
MSE = 2 + 0.25
2.25.
MSE
0.05.
bias = −σ²/n
−0.4.
MSE = Var
Bias zero.
MSE(θ̂) = E[(θ̂ − θ)²] = Var(θ̂) + bias(θ̂)². A small bias can be worth accepting if it reduces the variance enough.
Var = 2, bias = 0.5. Find the MSE.
Var = 0.04, bias = 0.1. Find the MSE.
σ² = 4, n = 10. Find −σ²/n.
Find the MSE with Var = 2 and bias 0.
Var + bias ✗
Var + bias² ✓
Square.
ignoring variance ✗
trade-off possible ✓
Bias–variance.
sd + bias ✗
variance + bias² ✓
Same units.
possible ✗
never negative ✓
Check.
Try each one on paper, then open the answer. Use the solver above to see the full working.
4
0.54
1.09
-3
0.25
8
Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.
Solve: …
E[(θ̂ − θ)²].
Var + bias².
No.
−σ²/n.
More in Advanced statistics and stochastic processes and the formula sheets.