Transition Matrix/Markov Process

Question

Let P be a 3x3 transition matrix for a markov process.

P = Row 1 [.7 .3 .4], Row 2 [.1 .6 .1] Row 3 [.2 .1 .5]

One can show that p⁴ = a 3x3 matrix with

row 1 [.552 .5167 .5439]

row 2 [.1875 .25 .1875]

row 3 [.2605 .2333 .2686]

(a) If we are in state 2 now, what is the probability that we will be in state 1 after one time interval?

(b) If we are in state 2 now, what will be the probability that we will be in state 1 after 4 time intervals?

Answer 1

Being in state 2 now is equivalent to x =

[0 1 0]^T

a)Then, the probability of being in state 1 after 1 time interval is P₁₂ the first row and second column of P.

It is .3

b)The probability of being in state 1 after 4 time intervals is from row 1 column 2 of P⁴

.5167

Note how the values in the rows, which are fairly far apart in P, are becoming close in P⁴