Markov Chain Need help!

Question

The weather on any given day in a particular city can be sunny, cloudy, or rainy. It has been observed to be predictable largely on the basis of the weather on the previous day. Specfically:

if it is sunny on one day, it will be sunny the next day 1/5 of the time, and be cloudy the next day 1/5 of the time

if it is cloudy on one day, it will be sunny the next day 3/5 of the time, and be cloudy the next day 1/5 of the time

if it is rainy on one day, it will be sunny the next day 2/5 of the time, and be cloudy the next day 3/5 of the time

Using 'sunny', 'cloudy', and 'rainy' (in that order) as the states in a system, set up the transition matrix for a Markov chain to describe this system.

Find the proportion of days that have each type of weather in the long run. I need a transformative 3x3 matrix and proportions of each weather(sunny, cloudy rainy) 

Answer 1

To build the transition matrix, each column represents the CURRENT state, and is filled with the (conditional) probabilities of the NEXT state. Make sure to keep the same order. For instance, the first column (sunny) contains:

P(Sunny Tomorrow | Sunny Today) = 1/5

P(Cloudy Tomorrow | Sunny Today) = 1/5

P(Rainy Tomorrow | Sunny Today) = ??

To compute the missing probability, use the fact that the sum of each column must be 1. The ?? above would thus be 3/5.

Do the same thing for the second column (cloudy) and third column (rainy). The transition matrix is:

1/5 3/5 2/5

1/5 1/5 3/5

3/5 1/5 0

Note that each of the numbers in the last row had to be computed.

---

The "long run" (steady-state) distribution vector x is obtained by solving the system Tx = x, or equivalently (T-I)x = 0. The matrix T-I is:

-4/5 3/5 2/5

1/5 -4/5 3/5

3/5 1/5 -1

I used technology to find the Reduced Row Echelon Form (you should know how to do this at this point in the course).

1 0 -17/13

0 1 -14/13

0 0 0

A useful check it that if you've done everything right so far, there MUST be a free variable. Here it is the third row (rainy), so we have the general solution:

Sunny - 17/13 * Rainy = 0

Cloudy - 14/13 * Rainy = 0

You can choose a convenient value of the free variable to get a particular solution. Rainy = 13 works since it clears denominators, given the solution (17, 14, 13).

In order to get the steady-state vector, the sum of the entries must be 1. Add the entries in the solution vector, and multiply by the reciprocal of that sum.

(17/44, 14/44, 13/44) = (17/44, 7/22, 13/44)

I STRONGLY RECOMMEND checking your solution by computing the product of the transition matrix and this steady-state vector; the result should be the steady-state vector (Tx = x).