Need help solving this question. Thanks in advance. 10 marks6. Consider a Markov
ID: 2922814 • Letter: N
Question
Need help solving this question. Thanks in advance.
10 marks6. Consider a Markov chain on f0,1,2) with transition matrix chain on [0,1,2) with transit Markov chain orn 0.3 0.2 0.5 P=1 0.5 0.1 0.41 . The process will eventually be absorbed at time T := min(n 01x,-2) Determine the probability that the process is absorbed in an odd number oj steps starting from state Xo = 0, i.e., determine P(T = odd|Xo = 0). Hint: Use an appropriate first-step analysis to set up a system of equations for P(T = oddlXo = i); i = 0,1.]Explanation / Answer
The process is initially at state 0 that is X0 =0 . Now if K if the probability that it will be absorbed in odd number of steps then (1-K ) will be the probability that it willl be absorbed in even number of steps ( because it eventaully have to get absorbed ) Similarly, from state 1, let L be the probability of getting absorbed in odd number of steps and then (1-L ) will be the probability of getting absorbed in even number of steps. Now taking the various cases here:
Therefore the equation for K from here we get is:
K = 0.5 + 0.3*(1 - K) + 0.2*(1 - L)
1.3K = 1 - 0.2L
Therefore, L = 5*(1 - 1.3K)
Now doing similar analysis from state 1, we will get the equation as:
L = 0.4 + 0.5*(1 - K) + 0.1*(1 - L)
1.1L = 1 - 0.5K
Now put L = 5*(1 - 1.3K) in the above equation to get:
1.1*5*(1 - 1.3K) = 1 - 0.5K
5.5 - 7.15K = 1 - 0.5K
K = 4.5 / 6.65 = 0.6767
Therefore 0.6767 is the required probability here.