Type I and Type II Errors, Power Explained with Worked Examples

A Type I error rejects a true (probability ); a Type II error accepts when is true (probability ); the power is , the probability of correctly rejecting .

Write 2x or 2*x to multiply, ^ for powers, and ; between simultaneous equations. You can also type factor, expand, derivative of, integrate … from a to b, limit … as x->0 or 15% of 80.

Try an example

Before you start

This page is easier if you are comfortable with:

What Type I and Type II errors, power means

Let and test against , rejecting when . Then . If the truth is , the power is and : the test is cautious (small ) but misses the effect more than half the time. The probabilities are binomial (the binomial distribution, the test for a binomial proportion).

There is a trade-off: lowering the threshold to raises to 0.0577 and the power to 0.608. Increasing the sample size improves both. For a normal mean the power depends on the effect size, the standard error and (the test for the mean of a normal, standardising).

The Neyman–Pearson lemma shows that likelihood-ratio tests have the highest power for a given (the Neyman–Pearson lemma, the likelihood-ratio statistic). Multiple tests inflate the overall Type I error (Bonferroni and Benjamini–Hochberg).

What is it used for?

Type I and Type II errors describe the two ways a hypothesis test can go wrong.

How to use type i and type ii errors, power

  1. Define the critical region (reject when ...).
  2. Compute under .
  3. Compute the probability of the critical region under : the power.
  4. power.

Graph

B(20, p) test rejecting for X ≥ 15: α = 0.021, power = 0.416, β = 0.584α: 0.021power: 0.416β: 0.58400.511.522.533.5400.10.20.30.40.50.60.7B(20, p) test rejecting for X ≥ 15: α = 0.021, power = 0.416, β = 0.584quantityyGraph with x from 0 to 4 and y from 0 to 0.7. Shows bars (3); point α: 0.021 at (1, 0.0207); point power: 0.416 at (2, 0.4164); point β: 0.584 at (3, 0.5836).

A small α comes with a large β for this sample size.

Worked examples

These examples are worked by hand, step by step, so you can compare your own working.

Example 1: Type I error

Find for .

Worked example
Answer
0.020690.02069
  1. Sum

Example 2: power

Find the power for .

Worked example
Answer
0.416370.41637
  1. Value

Example 3: Type II error

Find .

Worked example
Answer
0.583630.58363
  1. Formula

Special cases

A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.

H₀: p = 0.5, X ~ B(10, p), reject if X ≥ 8

α = 56/1024 ≈ 0.0547

Type I error rate.

Power at p = 0.7

P(X ≥ 8) ≈ 0.3828

Chance of rejecting when p = 0.7.

Type II error

β = 1 − power ≈ 0.6172

Failing to reject.

Trade-off

lowering α raises β

For fixed n.

Word problems

A Type I error is rejecting a true H₀ (probability α = significance level). A Type II error is failing to reject a false H₀ (probability β). The power of the test is 1 − β.

Type I probability

Reject H₀: p = 0.5 if X ≥ 8 with X ~ B(10, p). Find α.

  1. (45 + 10 + 1)/1024
  2. Check: 0.0547 ✓0.0547.

Power

Find the power when the true p is 0.7.

  1. P(X ≥ 8 | p = 0.7)
  2. Check: 0.3828 ✓0.3828.

Type II probability Higher

Find β at p = 0.7.

  1. 1 − 0.3828
  2. Check: 0.6172 ✓0.6172.

Larger p Higher

Is the power larger at p = 0.9? Find P(X ≥ 8).

  1. 0.9 gives more successes
  2. Check: 0.9298 ✓0.9298.

Common mistakes

Swapping α and β

α = P(accept H₀ | false) ✗

α = P(reject H₀ | true) ✓

Type I is rejecting a true null.

Power as 1 − α

power = 0.9453 ✗

power = 1 − β ✓

Power depends on the true p.

Using H₀ to find power

p = 0.5 ✗

use the alternative p ✓

Power is computed under H₁.

Ignoring the critical region

P(X = 8) ✗

P(X ≥ 8) ✓

Include the whole region.

Exam tips

  1. α: reject true H₀.
  2. β: keep false H₀.
  3. Power = 1 − β.
  4. Compute under H₁.
  5. Bigger n raises power.

Practice questions

Try each one on paper, then open the answer. Use the solver above to see the full working.

1. α for reject X ≥ 9, B(10, 0.5) Foundation

0.0107

2. Power at p = 0.9 for X ≥ 9 Foundation

0.7361

3. β = 1 − 0.7361 Foundation

0.2639

4. P(X = 10) at p = 0.5 Higher

0.001

5. P(X = 10) at p = 0.8 Higher

0.1074

6. Power as 1 − β when β = 0.3 Higher

0.7

Practice generator

Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.

Solve: …

Key terms

Type I error
Rejecting a true null hypothesis.
Type II error
Not rejecting a false null.
Significance level (α)
P(Type I).
Power
1 − β.
Critical region
Values leading to rejection.
Alternative hypothesis
H₁.

Frequently asked questions

What is a Type I error?

Rejecting H₀ when it is true.

What is a Type II error?

Failing to reject H₀ when it is false.

What is power?

The probability of correctly rejecting a false H₀.

How do α and β relate?

Lowering α usually raises β.

Which page covers the binomial distribution?

binomial distribution.

Which page covers Yates’ correction?

Yates’ continuity correction.

More in Advanced statistics and stochastic processes and the formula sheets.