Markov Chain Problem

Question

An investor bought shares of a company at $30 per share. Currently the value of the share is $31. The investor will sell the shares either as soon as he makes 10% or more, or when the stock drops to $30. At the end of each trading day the stock gains $2 with probability (w.p.) 0.1, $1 w.p. 0.3, remains the same w.p. 0.3 and loses $1 w.p. 0.3.

(a) Model this process as Markov Chain and classify its states as transient or recurrent.

(b) What is the probability of making at least 10% ?

(c) What is the probability of gaining nothing at the end if the next trading day yields a gain of $1?

(d) What is the expected number of days that the value of the share is $31, before the investor sells it?

Answer 1

Let being at $30 be state 1, $31 be state 2, $32 be state 3, and $33/$34 as state 4 (as the problem never differentiates between these two

Then, M =

1 0 0 0

.3 .3 .3 .1

0 .3 .3 .4

0 0 0 1

The process is recurrent, because it has absorbing states at $30 and $33/34 and positive probabilities of reaching absorbing states from the other states.

M^∞ =

1 0 0 0

.525 0 0 .475

.225 0 0 .775

0 0 0 1

You can calculate this by repeatedly squaring the transition matrix (the easiest way) or using other methods.

The probability of gaining 10% is .475 (see row 2, column 4 above)

If you initially gain, you are then in the third row. The probability of then gaining nothing is row 3, column 1 equals .225

To get the expected number of times to return to $31, we look at Q, the non-absorbing part of the matrix above. It is

.3 .3

.3 .3

I - Q is

.7 -.3

-.3 .7

and (I - Q)^-1 =

1.75 .75

..75 1.75

Starting at $31, we expect it to be at $31 1.75 times. (This includes day 0).