Suppose we have the probability matrix with states 1, 2, 3 Then the Matrix is P

ID: 2969404 • Letter: S

Question

Suppose we have the probability matrix with states 1, 2, 3

Then the Matrix is

P =

Current state After State;

1 2 3

1 [ 1/3 1/3 1/3

2 0 1/2 1/2

3 1/4 3/4 0 ]

Starting at State 1, what is the mean time that the process spends in state 2 prior to first hitting state 3?

Starting at State 1, what is the mean time that the process spends in state 2 prior to returning to state 1?

Comment: I have no idea how to start a state return problem. Also how does the initial state affect the answer... suppose that I asked...

Starting at State i, what is the mean time that the process spend in state 2 prior to returning to state 1? given that state i =/= 2.

Thank you.

The mean time it spends in 2 before going to 3, as it has to go to 3 after 2, is 1/(1-p) = 1/(1/2) = 2

The mean time in 2 before going to 1.

Note that it goes through 3, where it has 3/4 chance of going back to 2 and 1/4 chance of going to 1.

Thus, it is simply 2/(1 - 3/4) = 2 * 4 = 8