Markov Chains

Question

Box 1 contains 3 red marbles and Box 2 contains 3 blue marbles. A marble is drawn at random from each box and these two marbles are interchanged (each is put in the other box). Let the states of this Markov Chain be the number of red marbles in Box 1. Find the transition matrix.

Answer 1

Case X_n = 0:

We must have X_n+1 = 1.

 

Case X_n = 1:

Choosing a red ball from box 1 (1/3 chance) and a blue one from box 2 (1/3 chance) gives X_n+1 = 0 (1/9 chance).

Choosing a blue ball from box 1 (2/3 chance) and a red one from box 2 (2/3 chance) gives X_n+1 = 2 (4/9 chance).

Otherwise we have X_n+1 = 1 (4/9 chance).

 

Case X_n = 2:
Choosing a red ball from box 1 (2/3 chance) and a blue one from box 2 (2/3 chance) gives X_n+1 = 1 (4/9 chance).
Choosing a blue ball from box 1 (1/3 chance) and a red one from box 2 (1/3 chance) gives X_n+1 = 3 (1/9 chance).
Otherwise we have X_n+1 = 2 (4/9 chance).

 

Case X_n = 3:
We must have X_n+1 = 2.

 

Thus the transition matrix is

 

      0     1     2     3

   ---------------------

0 |  0     1     0     0

1 |1/9  4/9  4/9    0

2 |  0   4/9  4/9  1/9

3 |  0     0     1     0