Maximum Likelihood Estimation Explained with Worked Examples

The maximum likelihood estimate is the value of the parameter that makes the observed data most probable: , usually found from .

Write 2x or 2*x to multiply, ^ for powers, and ; between simultaneous equations. You can also type factor, expand, derivative of, integrate … from a to b, limit … as x->0 or 15% of 80.

Try an example

Before you start

This page is easier if you are comfortable with:

What Maximum likelihood estimation means

For 7 heads in 10 tosses, and ; setting the derivative gives , and at the maximum. For Poisson counts : , so . Differentiating a logarithm is covered in the derivative of ln x, and the likelihood of independent data is a product, so taking logs turns it into a sum.

The principle gives the sample proportion for a binomial (the binomial distribution), the sample mean for a Poisson (the Poisson distribution), and closed forms for the exponential and normal cases in the exponential rate and the normal mean and variance. A second derivative test confirms a maximum (stationary points).

MLEs are consistent and asymptotically normal, with variance given by the inverse Fisher information (Fisher information). Competing models are compared by likelihood ratios (the likelihood ratio test statistic) or by penalised criteria (AIC and BIC). In Bayesian terms the MLE is the mode of the posterior with a flat prior (posterior ∝ likelihood × prior).

What is it used for?

Maximum likelihood chooses the parameter value under which the observed data would be most probable.

How to use maximum likelihood estimation

  1. Write the likelihood and its logarithm.
  2. Differentiate with respect to and set to zero.
  3. Solve for and check it is a maximum.
  4. Evaluate the maximised log-likelihood if needed.

Graph

Likelihood p⁷(1 − p)³ for 7 heads in 10 tosses, maximum at 0.7maximum at p = 0.700.10.20.30.40.50.60.70.80.9100.00050.0010.00150.0020.0025Likelihood p⁷(1 − p)³ for 7 heads in 10 tosses, maximum at 0.7plikelihoodL(p) ∝ p⁷(1 − p)³Graph with x from 0 to 1 and y from 0 to 0.0025. Shows the curve L(p) ∝ p⁷(1 − p)³; point maximum at p = 0.7 at (0.7, 0.002223).

The likelihood is largest at the observed proportion 0.7.

Worked examples

These examples are worked by hand, step by step, so you can compare your own working.

Example 1: a proportion

Find the MLE of for 7 heads in 10 tosses.

Worked example
Answer
p^=0.7\hat p=0.7
  1. Equation

Example 2: a Poisson mean

Find the MLE of for the data .

Worked example
Answer
λ^=3\hat\lambda=3
  1. Sample mean

Example 3: maximised log-likelihood

Find at .

Worked example
Answer
−6.1086-6.1086
  1. Terms

Special cases

A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.

7 successes in 10

p̂ = 7/10

0.7.

Log-likelihood at p̂

7 ln 0.7 + 3 ln 0.3

−6.1086.

Poisson data (2, 3, 4, 3)

λ̂ = x̄

3.

Log-likelihood at p = 0.5

10 ln 0.5

−6.9315.

Word problems

The MLE is the parameter value maximising L(θ), found by solving d(ln L)/dθ = 0. For binomial data p̂ = k/n; for Poisson data λ̂ = x̄.

Binomial

7 successes in 10 trials. Find p̂.

  1. 7/10
  2. Check: ✓0.7

Log-likelihood

Evaluate 7 ln p + 3 ln(1 − p) at p̂ = 0.7.

  1. −2.4965 − 3.6119
  2. Check: ✓-6.1086

Poisson Higher

Data 2, 3, 4, 3. Find λ̂.

  1. 12/4
  2. Check: ✓3

Comparison Higher

Log-likelihood at p = 0.5.

  1. 10 ln 0.5
  2. Check: lower than at 0.7 ✓-6.9315

Common mistakes

Maximising L without logs

harder ✗

maximise ln L ✓

Same maximiser.

Likelihood = probability of θ

confusing ✗

probability of the data ✓

Function of θ.

Ignoring second derivative

assuming a maximum ✗

check concavity ✓

Could be a minimum.

Boundary maxima

always interior ✗

check the boundary ✓

e.g. uniform.

Exam tips

  1. Take logs.
  2. Differentiate.
  3. Solve = 0.
  4. Check it is a maximum.
  5. MLE invariance.

Practice questions

Try each one on paper, then open the answer. Use the solver above to see the full working.

1. p̂ for 3 successes in 12 Foundation

0.25

2. p̂ for 45 in 60 Foundation

0.75

3. λ̂ for data 1, 2, 3 Foundation

2

4. ln L at p = 0.5 for 6 of 12 to 4 d.p. Higher

-8.3178

5. ln L at p̂ = 0.25 for 3 of 12 to 4 d.p. Higher

-6.748

6. λ̂ for 5, 5, 8, 2 Higher

5

Practice generator

Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.

Solve: …

Key terms

Likelihood
L(θ) = P(data
Log-likelihood
ln L.
Maximum likelihood estimate
Maximiser.
Score equation
d ln L/dθ = 0.
Invariance
MLE of g(θ) is g(θ̂).
Observed data
Fixed in L.

Frequently asked questions

What is maximum likelihood?

Choosing θ that makes the observed data most probable.

Why take logs?

Products become sums.

What is the MLE for a binomial p?

k/n.

What is the MLE for a Poisson λ?

x̄.

Which page covers the exponential MLE?

MLE for the exponential rate.

Which page covers the normal MLE?

MLE for the normal.

More in Advanced statistics and stochastic processes and the formula sheets.