markov chains

Question

If the animal is in the woods on one observation, then it is three times as likely to be in the woods as the meadows on the next observation. If the animal is in the meadows on one observation, then it is twice as likely to be in the meadows as the woods on the next observation.

Assume that state 1 is being in the meadows and that state 2 is being in the woods.

(1) Find the transition matrix for this Markov process.

 

(2) If the animal is initially in the woods, what is the probability that it is in the woods on the next three observations?

 

(3) If the animal is initially in the woods, what is the probability that it is in the meadow on the next three observations? 

Answer 1

 

Hi Katie,

Let S_i, i = 1, 2, denote the state i,

where i = 1 and i = 2 are for meadows and woods, respectively.
 
If T_ij denotes the transition from S_j to S_i,
transition probabilities are given as follows.
 
     Prob(T₁₁) = 2/3
 
     Prob(T₂₁) = 1- Prob(T₁₁) = 1 - 2/3 = 1/3
 
     Prob(T₁₂) = 1/4 = 1- Prob(T₂₂) = 1 - 3/4 = 1/4
 
     Prob(T₂₂) = 3/4

 

(1) So, the Markovian transition matrix, M, is

 

     ⌈ 2/3     1/4 ⌉

     ⌊ 1/3     3/4 ⌋

 

(2) M³ = M²·M

 

           = ⌈ 19/36   17/48 ⌉ · ⌈ 2/3   1/4 ⌉

              ⌊ 17/36   31/48 ⌋   ⌊ 1/3   3/4 ⌋

 

           = ⌈ 203/432   229/576 ⌉

              ⌊ 229/432   347/576 ⌋

 

     M³₂₂ from the matrix M³ represents the probability of the given event,

     which is 0.6024 (= 347/576).

 

(3) M³₁₂ from the matrix M³ represents the probability of the given event,

     which is 0.3976 (= 229/576).