markov chains

Question

(i) If volume is high this week, then next week it will be high with a probability of 0.9 and low with a probability of 0.1.
(ii) If volume is low this week then it will be high next week with a probability of 0.3.

Assume that state 1 is high volume and that state 2 is low volume.

(1) Find the transition matrix for this Markov process.

 

(2) If the volume this week is high, what is the probability that the volume will be high two weeks from now?

 

(3) What is the probability that volume will be high for three consecutive weeks?

Answer 1

 

 

(1) Q_ij = P(X_n+1 = j | X_n = i) =

 

       1      2

1  ⌈ 0.9   0.1 ⌉

2  ⌊ 0.3   0.7 ⌋

 

 

(2) P(X_n+2 = 1 | X_n = 1)

 

= P(X_n+2 = 1 | X_n+1 = 1)P(X_n+1 = 1 | X_n = 1) + P(X_n+2 = 1 | X_n+1 = 2)P(X_n+1 = 2 | X_n = 1)

 

= 0.9² + (0.3)(0.1) = 0.81 + 0.03 = 0.84

 

 

(3) The long term probability for three consecutive weeks to all have high volume requires us to get the stationary distribution π_i = P(X_n = i)

 

πQ = π gives equations

 

0.9π₁ + 0.3π₂ = π₁, and π₁ + π₂ = 1

 

Solving this system gives π₁ = 0.75 and π₂ = 0.25

 

Now we are ready to answer the question.

 

P(X_n+3 = X_n+2 = X_n+1 = 1)

 

= P(X_n+3 = 1 | X_n+2 = 1)P(X_n+2 = 1 | X_n+1 = 1)P(X_n+1 = 1 | X_n = 1)P(X_n = 1)

+ P(X_n+3 = 1 | X_n+2 = 1)P(X_n+2 = 1 | X_n+1 = 1)P(X_n+1 = 1 | X_n = 2)P(X_n = 2)

 

= 0.9³(0.75) + 0.9²(0.3)(0.25)

 

= 0.6075