How to use the Recursive method to calculate the average number of times that the Markov chain visits a given state ?

Question

Dear expert,

In the insight of what mentioned in the appendix of this article,

https://link.springer.com/article/10.1007/s11538-011-9638-4

My question is, if we already have the transition probability matrix P, how could we calculate numerically v^=α0(I−P)−1, as the limit of the recurrence v(t+1)=v(t)P+α0?

Answer 1

You can rewrite v(t) = α₀(I + IP + IP^2 + IP^3 + ...) or α₀(I + P + P^2 + P^3 + ...)

Note that (I - P)(I + P + P^2 + P^3 + ... + P^n) = I - P^n+1

As these are transient states, lim n -> ∞ Pⁿ = 0, and lim n -> ∞ I - P^n+1 = I

Thus, (I + P + P^2 + P^3 + ... + P^n + ...) = (I - P)^-1

Thus, α₀(I + P + P^2 + P^3 + ...) = α₀(I - P)^-1

lim t -> ∞ v(t) = α₀(I - P)^-1