P = (0.9 0.1; 0.5 0.5)
P²(1,1) = 0.9·0.9 + 0.1·0.5
0.86.
The -step transition probabilities are the entries of the matrix power , and : to go in steps, go in steps to some state and then steps from there.
This page is easier if you are comfortable with:
For : , so , summing over the intermediate state with the law of total probability. Squaring again, , and .
As grows the rows of approach the stationary distribution (the stationary distribution): already has rows and . The matrix products are computed as in matrix multiplication; for large powers use diagonalisation (diagonalisation, eigenvalues of a 2 × 2 matrix).
The equation is the Markov (memoryless) property combined with total probability: the multiplication rule (the multiplication rule) for each path and addition over intermediate states. Its continuous-time counterpart is the Kolmogorov equations, and it also appears in the Poisson process and Brownian motion.
The Chapman–Kolmogorov equation says that multi-step probabilities come from multiplying the transition matrix by itself.
Each row of P⁴ adds to 1 and is close to the stationary distribution.
These examples are worked by hand, step by step, so you can compare your own working.
Find .
Find .
Find .
A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.
P²(1,1) = 0.9·0.9 + 0.1·0.5
0.86.
0.9·0.1 + 0.1·0.5
0.14.
0.5·0.9 + 0.5·0.5
0.70.
P²(1,1) + P²(1,2)
1.
P⁽ⁿ⁺ᵐ⁾ = P⁽ⁿ⁾P⁽ᵐ⁾: go n steps to an intermediate state, then m steps. The n-step probabilities are the entries of Pⁿ.
Find P²(1,1).
Find P²(1,2).
Find P²(2,1).
Check P²(1,1) + P²(1,2).
entrywise squares ✗
matrix product ✓
Sum over intermediates.
only one middle state ✗
sum over all ✓
Chapman–Kolmogorov.
check columns ✗
rows sum to 1 ✓
Row-stochastic.
PP is symmetric ✗
P⁽ⁿ⁾P⁽ᵐ⁾ commutes for powers ✓
Same matrix.
Try each one on paper, then open the answer. Use the solver above to see the full working.
0.72
0.28
0.56
0.44
1
0.5
Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.
Solve: …
P⁽ⁿ⁺ᵐ⁾ = P⁽ⁿ⁾P⁽ᵐ⁾.
Square the transition matrix.
Any intermediate state may be visited.
1.
stationary distribution.
Poisson process.
More in Advanced statistics and stochastic processes and the formula sheets.