H₀: p = 0.5, X ~ B(10, p), reject if X ≥ 8
α = 56/1024 ≈ 0.0547
Type I error rate.
A Type I error rejects a true (probability ); a Type II error accepts when is true (probability ); the power is , the probability of correctly rejecting .
This page is easier if you are comfortable with:
Let and test against , rejecting when . Then . If the truth is , the power is and : the test is cautious (small ) but misses the effect more than half the time. The probabilities are binomial (the binomial distribution, the test for a binomial proportion).
There is a trade-off: lowering the threshold to raises to 0.0577 and the power to 0.608. Increasing the sample size improves both. For a normal mean the power depends on the effect size, the standard error and (the test for the mean of a normal, standardising).
The Neyman–Pearson lemma shows that likelihood-ratio tests have the highest power for a given (the Neyman–Pearson lemma, the likelihood-ratio statistic). Multiple tests inflate the overall Type I error (Bonferroni and Benjamini–Hochberg).
Type I and Type II errors describe the two ways a hypothesis test can go wrong.
A small α comes with a large β for this sample size.
These examples are worked by hand, step by step, so you can compare your own working.
Find for .
Find the power for .
Find .
A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.
α = 56/1024 ≈ 0.0547
Type I error rate.
P(X ≥ 8) ≈ 0.3828
Chance of rejecting when p = 0.7.
β = 1 − power ≈ 0.6172
Failing to reject.
lowering α raises β
For fixed n.
A Type I error is rejecting a true H₀ (probability α = significance level). A Type II error is failing to reject a false H₀ (probability β). The power of the test is 1 − β.
Reject H₀: p = 0.5 if X ≥ 8 with X ~ B(10, p). Find α.
Find the power when the true p is 0.7.
Find β at p = 0.7.
Is the power larger at p = 0.9? Find P(X ≥ 8).
α = P(accept H₀ | false) ✗
α = P(reject H₀ | true) ✓
Type I is rejecting a true null.
power = 0.9453 ✗
power = 1 − β ✓
Power depends on the true p.
p = 0.5 ✗
use the alternative p ✓
Power is computed under H₁.
P(X = 8) ✗
P(X ≥ 8) ✓
Include the whole region.
Try each one on paper, then open the answer. Use the solver above to see the full working.
0.0107
0.7361
0.2639
0.001
0.1074
0.7
Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.
Solve: …
Rejecting H₀ when it is true.
Failing to reject H₀ when it is false.
The probability of correctly rejecting a false H₀.
Lowering α usually raises β.
binomial distribution.
More in Advanced statistics and stochastic processes and the formula sheets.