Transition probability matrix of a Markov chain.












1












$begingroup$


I am not understanding how is the transition probability matrix of the following example constructed.



Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$; if it rained today but not yesterday, then it will rain tomorrow with probability $0.5$; if it rained yesterday but not today, then it will rain tomorrow with probability $0.4$; if it has not rained in the past two days, then it will rain tomorrow with probability $0.2$.



Let



state $0$ if it rained both today and yesterday,



state $1$ if it rained today but not yesterday,



state $2$ if it rained yesterday but not today,



state $3$ if it did not rain either yesterday or today.



The preceding would then represent a four-state Markov chain having a transition
probability matrix



$$P=
begin{bmatrix}
0.7 & 0 & 0.3 & 0 \
0.5 & 0 & 0.5 & 0 \
0 & 0.4 & 0 & 0.6 \
0 & 0.2 & 0 & 0.8 \
end{bmatrix}.
$$



Why is $P_{10}=0.5$ ? As the 2nd row corresponding to state $1$ represents it rained today but not yesterday, can't i assign the $0.5$ in $P_{11}$ or in $P_{13}$ ?



$bullet$ Second portion of the example is : Given that it rained on Monday and
Tuesday, what is the probability that it will rain on Thursday?



For solving this we have to compute two-step transition matrix. But if I were asked to find the probability that it will rain on Friday, would I have to compute three-step transition matrix ? Will I count the step from Tuesday?



And the book says that rain on Thursday is equivalent to the process being in either state $0$ or state $1$. Why it is not other states rather than state $0$ or state $1$?










share|cite|improve this question











$endgroup$

















    1












    $begingroup$


    I am not understanding how is the transition probability matrix of the following example constructed.



    Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$; if it rained today but not yesterday, then it will rain tomorrow with probability $0.5$; if it rained yesterday but not today, then it will rain tomorrow with probability $0.4$; if it has not rained in the past two days, then it will rain tomorrow with probability $0.2$.



    Let



    state $0$ if it rained both today and yesterday,



    state $1$ if it rained today but not yesterday,



    state $2$ if it rained yesterday but not today,



    state $3$ if it did not rain either yesterday or today.



    The preceding would then represent a four-state Markov chain having a transition
    probability matrix



    $$P=
    begin{bmatrix}
    0.7 & 0 & 0.3 & 0 \
    0.5 & 0 & 0.5 & 0 \
    0 & 0.4 & 0 & 0.6 \
    0 & 0.2 & 0 & 0.8 \
    end{bmatrix}.
    $$



    Why is $P_{10}=0.5$ ? As the 2nd row corresponding to state $1$ represents it rained today but not yesterday, can't i assign the $0.5$ in $P_{11}$ or in $P_{13}$ ?



    $bullet$ Second portion of the example is : Given that it rained on Monday and
    Tuesday, what is the probability that it will rain on Thursday?



    For solving this we have to compute two-step transition matrix. But if I were asked to find the probability that it will rain on Friday, would I have to compute three-step transition matrix ? Will I count the step from Tuesday?



    And the book says that rain on Thursday is equivalent to the process being in either state $0$ or state $1$. Why it is not other states rather than state $0$ or state $1$?










    share|cite|improve this question











    $endgroup$















      1












      1








      1


      3



      $begingroup$


      I am not understanding how is the transition probability matrix of the following example constructed.



      Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$; if it rained today but not yesterday, then it will rain tomorrow with probability $0.5$; if it rained yesterday but not today, then it will rain tomorrow with probability $0.4$; if it has not rained in the past two days, then it will rain tomorrow with probability $0.2$.



      Let



      state $0$ if it rained both today and yesterday,



      state $1$ if it rained today but not yesterday,



      state $2$ if it rained yesterday but not today,



      state $3$ if it did not rain either yesterday or today.



      The preceding would then represent a four-state Markov chain having a transition
      probability matrix



      $$P=
      begin{bmatrix}
      0.7 & 0 & 0.3 & 0 \
      0.5 & 0 & 0.5 & 0 \
      0 & 0.4 & 0 & 0.6 \
      0 & 0.2 & 0 & 0.8 \
      end{bmatrix}.
      $$



      Why is $P_{10}=0.5$ ? As the 2nd row corresponding to state $1$ represents it rained today but not yesterday, can't i assign the $0.5$ in $P_{11}$ or in $P_{13}$ ?



      $bullet$ Second portion of the example is : Given that it rained on Monday and
      Tuesday, what is the probability that it will rain on Thursday?



      For solving this we have to compute two-step transition matrix. But if I were asked to find the probability that it will rain on Friday, would I have to compute three-step transition matrix ? Will I count the step from Tuesday?



      And the book says that rain on Thursday is equivalent to the process being in either state $0$ or state $1$. Why it is not other states rather than state $0$ or state $1$?










      share|cite|improve this question











      $endgroup$




      I am not understanding how is the transition probability matrix of the following example constructed.



      Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$; if it rained today but not yesterday, then it will rain tomorrow with probability $0.5$; if it rained yesterday but not today, then it will rain tomorrow with probability $0.4$; if it has not rained in the past two days, then it will rain tomorrow with probability $0.2$.



      Let



      state $0$ if it rained both today and yesterday,



      state $1$ if it rained today but not yesterday,



      state $2$ if it rained yesterday but not today,



      state $3$ if it did not rain either yesterday or today.



      The preceding would then represent a four-state Markov chain having a transition
      probability matrix



      $$P=
      begin{bmatrix}
      0.7 & 0 & 0.3 & 0 \
      0.5 & 0 & 0.5 & 0 \
      0 & 0.4 & 0 & 0.6 \
      0 & 0.2 & 0 & 0.8 \
      end{bmatrix}.
      $$



      Why is $P_{10}=0.5$ ? As the 2nd row corresponding to state $1$ represents it rained today but not yesterday, can't i assign the $0.5$ in $P_{11}$ or in $P_{13}$ ?



      $bullet$ Second portion of the example is : Given that it rained on Monday and
      Tuesday, what is the probability that it will rain on Thursday?



      For solving this we have to compute two-step transition matrix. But if I were asked to find the probability that it will rain on Friday, would I have to compute three-step transition matrix ? Will I count the step from Tuesday?



      And the book says that rain on Thursday is equivalent to the process being in either state $0$ or state $1$. Why it is not other states rather than state $0$ or state $1$?







      probability markov-chains






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Oct 6 '15 at 17:51







      ABC

















      asked Oct 6 '15 at 17:31









      ABCABC

      6141926




      6141926






















          2 Answers
          2






          active

          oldest

          votes


















          1












          $begingroup$

          First part:



          Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = text{(rain, rain)}$. Then we have the sequence $(a,b,c) = text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.



          Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = text{(no rain, rain)}$ and $(b,c) = text{(no rain, rain)}$, which can't happen.





          Second part:



          Notice that we start from state $0$, thus $pi(0) = begin{bmatrix} 1& 0 & 0 & 0end{bmatrix}$ and we are going to evaluate the probability:
          $$pi(0)cdot P^2 = begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18end{bmatrix}. $$
          Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).





          Third part:



          From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold:
          $$(a,b,c,d) = text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.



          1: $(a,b,c,d) = text{(rain, rain, rain, rain)}$



          2: $(a,b,c,d) = text{(rain, rain, no rain, rain)}$



          Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.



          Speaking with term of states the first $4-tuple$ corresponds to the path $0to 0to 0$, thus we have $p_{00}cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0to 2to 1$, thus $p_{02}cdot p_{21} = 0.3 cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.






          share|cite|improve this answer











          $endgroup$





















            1












            $begingroup$

            1) Refer to the probability tree diagram:



            $hspace{3cm}$enter image description here



            Let (yesterday, today): $S_0$: (R, R); $S_1$: (N, R); $S_2$: (R, N); $S_3$: (N, N).



            Then (yesterday, today, tomorrow):
            $$begin{align}
            (R,R,R) &Rightarrow S_0 stackrel{0.7}{to} S_0 Rightarrow P_{00}\
            (R,R,N) &Rightarrow S_0stackrel{0.3}{to} S_2 Rightarrow P_{02}\
            (R,N,R) &Rightarrow S_2stackrel{0.4}{to} S_1 Rightarrow P_{21}\
            (R,N,N) &Rightarrow S_2stackrel{0.6}{to} S_3 Rightarrow P_{23}\
            (N,R,R) &Rightarrow S_1stackrel{0.5}{to} S_0 Rightarrow P_{10}\
            (N,R,N) &Rightarrow S_1stackrel{0.5}{to} S_2 Rightarrow P_{12}\
            (N,N,R) &Rightarrow S_3stackrel{0.2}{to} S_1 Rightarrow P_{31}\
            (N,N,N) &Rightarrow S_3stackrel{0.8}{to} S_3 Rightarrow P_{33}\
            end{align}\
            P_{01}=P_{03}=P_{11}=P_{13}=P_{30}=P_{32}=P_{40}=P_{42}=0.$$
            Hence:
            $$P=
            begin{bmatrix}
            0.7 & 0 & 0.3 & 0 \
            0.5 & 0 & 0.5 & 0 \
            0 & 0.4 & 0 & 0.6 \
            0 & 0.2 & 0 & 0.8 \
            end{bmatrix}.$$
            2) Probability of raining on Thurday given that it rained on Monday and Tuesday:
            $$mathbb P(underbrace{RR}_{S_0}underbrace{RR}_{S_0})+mathbb P(underbrace{RR}_{S_0}underbrace{NR}_{S_1})=0.7cdot 0.7+0.3cdot 0.4=0.61.$$
            which is:
            $$begin{bmatrix}
            underbrace{1}_{S_0} & underbrace{0}_{S_1} & underbrace{0}_{S_2} & underbrace{0}_{S_3}
            end{bmatrix}
            begin{bmatrix}
            0.7 & 0 & 0.3 & 0 \
            0.5 & 0 & 0.5 & 0 \
            0 & 0.4 & 0 & 0.6 \
            0 & 0.2 & 0 & 0.8 \
            end{bmatrix}^2=
            begin{bmatrix}
            underbrace{0.49}_{S_0} & underbrace{0.12}_{S_1} & underbrace{0.21}_{S_2} & underbrace{0.18}_{S_3}end{bmatrix} Rightarrow\
            mathbb P(S_0)+mathbb P(S_1)=0.49+0.12=0.61.$$
            3) Probability of raining on Friday given that it rained on Monday and Tuesday. Can you figure this out both ways: probability tree diagram and markov chain formula? The initial state matrix is $(1,0,0,0)$, which must be multiplied by the cube of the transition matrix and the probabilities of $S_0$ and $S_1$ must be added.






            share|cite|improve this answer









            $endgroup$













              Your Answer





              StackExchange.ifUsing("editor", function () {
              return StackExchange.using("mathjaxEditing", function () {
              StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
              StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
              });
              });
              }, "mathjax-editing");

              StackExchange.ready(function() {
              var channelOptions = {
              tags: "".split(" "),
              id: "69"
              };
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function() {
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled) {
              StackExchange.using("snippets", function() {
              createEditor();
              });
              }
              else {
              createEditor();
              }
              });

              function createEditor() {
              StackExchange.prepareEditor({
              heartbeatType: 'answer',
              autoActivateHeartbeat: false,
              convertImagesToLinks: true,
              noModals: true,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: 10,
              bindNavPrevention: true,
              postfix: "",
              imageUploader: {
              brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
              contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
              allowUrls: true
              },
              noCode: true, onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              });


              }
              });














              draft saved

              draft discarded


















              StackExchange.ready(
              function () {
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1467415%2ftransition-probability-matrix-of-a-markov-chain%23new-answer', 'question_page');
              }
              );

              Post as a guest















              Required, but never shown

























              2 Answers
              2






              active

              oldest

              votes








              2 Answers
              2






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes









              1












              $begingroup$

              First part:



              Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = text{(rain, rain)}$. Then we have the sequence $(a,b,c) = text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.



              Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = text{(no rain, rain)}$ and $(b,c) = text{(no rain, rain)}$, which can't happen.





              Second part:



              Notice that we start from state $0$, thus $pi(0) = begin{bmatrix} 1& 0 & 0 & 0end{bmatrix}$ and we are going to evaluate the probability:
              $$pi(0)cdot P^2 = begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18end{bmatrix}. $$
              Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).





              Third part:



              From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold:
              $$(a,b,c,d) = text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.



              1: $(a,b,c,d) = text{(rain, rain, rain, rain)}$



              2: $(a,b,c,d) = text{(rain, rain, no rain, rain)}$



              Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.



              Speaking with term of states the first $4-tuple$ corresponds to the path $0to 0to 0$, thus we have $p_{00}cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0to 2to 1$, thus $p_{02}cdot p_{21} = 0.3 cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.






              share|cite|improve this answer











              $endgroup$


















                1












                $begingroup$

                First part:



                Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = text{(rain, rain)}$. Then we have the sequence $(a,b,c) = text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.



                Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = text{(no rain, rain)}$ and $(b,c) = text{(no rain, rain)}$, which can't happen.





                Second part:



                Notice that we start from state $0$, thus $pi(0) = begin{bmatrix} 1& 0 & 0 & 0end{bmatrix}$ and we are going to evaluate the probability:
                $$pi(0)cdot P^2 = begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18end{bmatrix}. $$
                Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).





                Third part:



                From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold:
                $$(a,b,c,d) = text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.



                1: $(a,b,c,d) = text{(rain, rain, rain, rain)}$



                2: $(a,b,c,d) = text{(rain, rain, no rain, rain)}$



                Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.



                Speaking with term of states the first $4-tuple$ corresponds to the path $0to 0to 0$, thus we have $p_{00}cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0to 2to 1$, thus $p_{02}cdot p_{21} = 0.3 cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.






                share|cite|improve this answer











                $endgroup$
















                  1












                  1








                  1





                  $begingroup$

                  First part:



                  Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = text{(rain, rain)}$. Then we have the sequence $(a,b,c) = text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.



                  Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = text{(no rain, rain)}$ and $(b,c) = text{(no rain, rain)}$, which can't happen.





                  Second part:



                  Notice that we start from state $0$, thus $pi(0) = begin{bmatrix} 1& 0 & 0 & 0end{bmatrix}$ and we are going to evaluate the probability:
                  $$pi(0)cdot P^2 = begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18end{bmatrix}. $$
                  Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).





                  Third part:



                  From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold:
                  $$(a,b,c,d) = text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.



                  1: $(a,b,c,d) = text{(rain, rain, rain, rain)}$



                  2: $(a,b,c,d) = text{(rain, rain, no rain, rain)}$



                  Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.



                  Speaking with term of states the first $4-tuple$ corresponds to the path $0to 0to 0$, thus we have $p_{00}cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0to 2to 1$, thus $p_{02}cdot p_{21} = 0.3 cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.






                  share|cite|improve this answer











                  $endgroup$



                  First part:



                  Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = text{(rain, rain)}$. Then we have the sequence $(a,b,c) = text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.



                  Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = text{(no rain, rain)}$ and $(b,c) = text{(no rain, rain)}$, which can't happen.





                  Second part:



                  Notice that we start from state $0$, thus $pi(0) = begin{bmatrix} 1& 0 & 0 & 0end{bmatrix}$ and we are going to evaluate the probability:
                  $$pi(0)cdot P^2 = begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18end{bmatrix}. $$
                  Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).





                  Third part:



                  From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold:
                  $$(a,b,c,d) = text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.



                  1: $(a,b,c,d) = text{(rain, rain, rain, rain)}$



                  2: $(a,b,c,d) = text{(rain, rain, no rain, rain)}$



                  Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.



                  Speaking with term of states the first $4-tuple$ corresponds to the path $0to 0to 0$, thus we have $p_{00}cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0to 2to 1$, thus $p_{02}cdot p_{21} = 0.3 cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.







                  share|cite|improve this answer














                  share|cite|improve this answer



                  share|cite|improve this answer








                  edited Oct 8 '15 at 1:53

























                  answered Oct 7 '15 at 16:58









                  thanasissdrthanasissdr

                  5,53811325




                  5,53811325























                      1












                      $begingroup$

                      1) Refer to the probability tree diagram:



                      $hspace{3cm}$enter image description here



                      Let (yesterday, today): $S_0$: (R, R); $S_1$: (N, R); $S_2$: (R, N); $S_3$: (N, N).



                      Then (yesterday, today, tomorrow):
                      $$begin{align}
                      (R,R,R) &Rightarrow S_0 stackrel{0.7}{to} S_0 Rightarrow P_{00}\
                      (R,R,N) &Rightarrow S_0stackrel{0.3}{to} S_2 Rightarrow P_{02}\
                      (R,N,R) &Rightarrow S_2stackrel{0.4}{to} S_1 Rightarrow P_{21}\
                      (R,N,N) &Rightarrow S_2stackrel{0.6}{to} S_3 Rightarrow P_{23}\
                      (N,R,R) &Rightarrow S_1stackrel{0.5}{to} S_0 Rightarrow P_{10}\
                      (N,R,N) &Rightarrow S_1stackrel{0.5}{to} S_2 Rightarrow P_{12}\
                      (N,N,R) &Rightarrow S_3stackrel{0.2}{to} S_1 Rightarrow P_{31}\
                      (N,N,N) &Rightarrow S_3stackrel{0.8}{to} S_3 Rightarrow P_{33}\
                      end{align}\
                      P_{01}=P_{03}=P_{11}=P_{13}=P_{30}=P_{32}=P_{40}=P_{42}=0.$$
                      Hence:
                      $$P=
                      begin{bmatrix}
                      0.7 & 0 & 0.3 & 0 \
                      0.5 & 0 & 0.5 & 0 \
                      0 & 0.4 & 0 & 0.6 \
                      0 & 0.2 & 0 & 0.8 \
                      end{bmatrix}.$$
                      2) Probability of raining on Thurday given that it rained on Monday and Tuesday:
                      $$mathbb P(underbrace{RR}_{S_0}underbrace{RR}_{S_0})+mathbb P(underbrace{RR}_{S_0}underbrace{NR}_{S_1})=0.7cdot 0.7+0.3cdot 0.4=0.61.$$
                      which is:
                      $$begin{bmatrix}
                      underbrace{1}_{S_0} & underbrace{0}_{S_1} & underbrace{0}_{S_2} & underbrace{0}_{S_3}
                      end{bmatrix}
                      begin{bmatrix}
                      0.7 & 0 & 0.3 & 0 \
                      0.5 & 0 & 0.5 & 0 \
                      0 & 0.4 & 0 & 0.6 \
                      0 & 0.2 & 0 & 0.8 \
                      end{bmatrix}^2=
                      begin{bmatrix}
                      underbrace{0.49}_{S_0} & underbrace{0.12}_{S_1} & underbrace{0.21}_{S_2} & underbrace{0.18}_{S_3}end{bmatrix} Rightarrow\
                      mathbb P(S_0)+mathbb P(S_1)=0.49+0.12=0.61.$$
                      3) Probability of raining on Friday given that it rained on Monday and Tuesday. Can you figure this out both ways: probability tree diagram and markov chain formula? The initial state matrix is $(1,0,0,0)$, which must be multiplied by the cube of the transition matrix and the probabilities of $S_0$ and $S_1$ must be added.






                      share|cite|improve this answer









                      $endgroup$


















                        1












                        $begingroup$

                        1) Refer to the probability tree diagram:



                        $hspace{3cm}$enter image description here



                        Let (yesterday, today): $S_0$: (R, R); $S_1$: (N, R); $S_2$: (R, N); $S_3$: (N, N).



                        Then (yesterday, today, tomorrow):
                        $$begin{align}
                        (R,R,R) &Rightarrow S_0 stackrel{0.7}{to} S_0 Rightarrow P_{00}\
                        (R,R,N) &Rightarrow S_0stackrel{0.3}{to} S_2 Rightarrow P_{02}\
                        (R,N,R) &Rightarrow S_2stackrel{0.4}{to} S_1 Rightarrow P_{21}\
                        (R,N,N) &Rightarrow S_2stackrel{0.6}{to} S_3 Rightarrow P_{23}\
                        (N,R,R) &Rightarrow S_1stackrel{0.5}{to} S_0 Rightarrow P_{10}\
                        (N,R,N) &Rightarrow S_1stackrel{0.5}{to} S_2 Rightarrow P_{12}\
                        (N,N,R) &Rightarrow S_3stackrel{0.2}{to} S_1 Rightarrow P_{31}\
                        (N,N,N) &Rightarrow S_3stackrel{0.8}{to} S_3 Rightarrow P_{33}\
                        end{align}\
                        P_{01}=P_{03}=P_{11}=P_{13}=P_{30}=P_{32}=P_{40}=P_{42}=0.$$
                        Hence:
                        $$P=
                        begin{bmatrix}
                        0.7 & 0 & 0.3 & 0 \
                        0.5 & 0 & 0.5 & 0 \
                        0 & 0.4 & 0 & 0.6 \
                        0 & 0.2 & 0 & 0.8 \
                        end{bmatrix}.$$
                        2) Probability of raining on Thurday given that it rained on Monday and Tuesday:
                        $$mathbb P(underbrace{RR}_{S_0}underbrace{RR}_{S_0})+mathbb P(underbrace{RR}_{S_0}underbrace{NR}_{S_1})=0.7cdot 0.7+0.3cdot 0.4=0.61.$$
                        which is:
                        $$begin{bmatrix}
                        underbrace{1}_{S_0} & underbrace{0}_{S_1} & underbrace{0}_{S_2} & underbrace{0}_{S_3}
                        end{bmatrix}
                        begin{bmatrix}
                        0.7 & 0 & 0.3 & 0 \
                        0.5 & 0 & 0.5 & 0 \
                        0 & 0.4 & 0 & 0.6 \
                        0 & 0.2 & 0 & 0.8 \
                        end{bmatrix}^2=
                        begin{bmatrix}
                        underbrace{0.49}_{S_0} & underbrace{0.12}_{S_1} & underbrace{0.21}_{S_2} & underbrace{0.18}_{S_3}end{bmatrix} Rightarrow\
                        mathbb P(S_0)+mathbb P(S_1)=0.49+0.12=0.61.$$
                        3) Probability of raining on Friday given that it rained on Monday and Tuesday. Can you figure this out both ways: probability tree diagram and markov chain formula? The initial state matrix is $(1,0,0,0)$, which must be multiplied by the cube of the transition matrix and the probabilities of $S_0$ and $S_1$ must be added.






                        share|cite|improve this answer









                        $endgroup$
















                          1












                          1








                          1





                          $begingroup$

                          1) Refer to the probability tree diagram:



                          $hspace{3cm}$enter image description here



                          Let (yesterday, today): $S_0$: (R, R); $S_1$: (N, R); $S_2$: (R, N); $S_3$: (N, N).



                          Then (yesterday, today, tomorrow):
                          $$begin{align}
                          (R,R,R) &Rightarrow S_0 stackrel{0.7}{to} S_0 Rightarrow P_{00}\
                          (R,R,N) &Rightarrow S_0stackrel{0.3}{to} S_2 Rightarrow P_{02}\
                          (R,N,R) &Rightarrow S_2stackrel{0.4}{to} S_1 Rightarrow P_{21}\
                          (R,N,N) &Rightarrow S_2stackrel{0.6}{to} S_3 Rightarrow P_{23}\
                          (N,R,R) &Rightarrow S_1stackrel{0.5}{to} S_0 Rightarrow P_{10}\
                          (N,R,N) &Rightarrow S_1stackrel{0.5}{to} S_2 Rightarrow P_{12}\
                          (N,N,R) &Rightarrow S_3stackrel{0.2}{to} S_1 Rightarrow P_{31}\
                          (N,N,N) &Rightarrow S_3stackrel{0.8}{to} S_3 Rightarrow P_{33}\
                          end{align}\
                          P_{01}=P_{03}=P_{11}=P_{13}=P_{30}=P_{32}=P_{40}=P_{42}=0.$$
                          Hence:
                          $$P=
                          begin{bmatrix}
                          0.7 & 0 & 0.3 & 0 \
                          0.5 & 0 & 0.5 & 0 \
                          0 & 0.4 & 0 & 0.6 \
                          0 & 0.2 & 0 & 0.8 \
                          end{bmatrix}.$$
                          2) Probability of raining on Thurday given that it rained on Monday and Tuesday:
                          $$mathbb P(underbrace{RR}_{S_0}underbrace{RR}_{S_0})+mathbb P(underbrace{RR}_{S_0}underbrace{NR}_{S_1})=0.7cdot 0.7+0.3cdot 0.4=0.61.$$
                          which is:
                          $$begin{bmatrix}
                          underbrace{1}_{S_0} & underbrace{0}_{S_1} & underbrace{0}_{S_2} & underbrace{0}_{S_3}
                          end{bmatrix}
                          begin{bmatrix}
                          0.7 & 0 & 0.3 & 0 \
                          0.5 & 0 & 0.5 & 0 \
                          0 & 0.4 & 0 & 0.6 \
                          0 & 0.2 & 0 & 0.8 \
                          end{bmatrix}^2=
                          begin{bmatrix}
                          underbrace{0.49}_{S_0} & underbrace{0.12}_{S_1} & underbrace{0.21}_{S_2} & underbrace{0.18}_{S_3}end{bmatrix} Rightarrow\
                          mathbb P(S_0)+mathbb P(S_1)=0.49+0.12=0.61.$$
                          3) Probability of raining on Friday given that it rained on Monday and Tuesday. Can you figure this out both ways: probability tree diagram and markov chain formula? The initial state matrix is $(1,0,0,0)$, which must be multiplied by the cube of the transition matrix and the probabilities of $S_0$ and $S_1$ must be added.






                          share|cite|improve this answer









                          $endgroup$



                          1) Refer to the probability tree diagram:



                          $hspace{3cm}$enter image description here



                          Let (yesterday, today): $S_0$: (R, R); $S_1$: (N, R); $S_2$: (R, N); $S_3$: (N, N).



                          Then (yesterday, today, tomorrow):
                          $$begin{align}
                          (R,R,R) &Rightarrow S_0 stackrel{0.7}{to} S_0 Rightarrow P_{00}\
                          (R,R,N) &Rightarrow S_0stackrel{0.3}{to} S_2 Rightarrow P_{02}\
                          (R,N,R) &Rightarrow S_2stackrel{0.4}{to} S_1 Rightarrow P_{21}\
                          (R,N,N) &Rightarrow S_2stackrel{0.6}{to} S_3 Rightarrow P_{23}\
                          (N,R,R) &Rightarrow S_1stackrel{0.5}{to} S_0 Rightarrow P_{10}\
                          (N,R,N) &Rightarrow S_1stackrel{0.5}{to} S_2 Rightarrow P_{12}\
                          (N,N,R) &Rightarrow S_3stackrel{0.2}{to} S_1 Rightarrow P_{31}\
                          (N,N,N) &Rightarrow S_3stackrel{0.8}{to} S_3 Rightarrow P_{33}\
                          end{align}\
                          P_{01}=P_{03}=P_{11}=P_{13}=P_{30}=P_{32}=P_{40}=P_{42}=0.$$
                          Hence:
                          $$P=
                          begin{bmatrix}
                          0.7 & 0 & 0.3 & 0 \
                          0.5 & 0 & 0.5 & 0 \
                          0 & 0.4 & 0 & 0.6 \
                          0 & 0.2 & 0 & 0.8 \
                          end{bmatrix}.$$
                          2) Probability of raining on Thurday given that it rained on Monday and Tuesday:
                          $$mathbb P(underbrace{RR}_{S_0}underbrace{RR}_{S_0})+mathbb P(underbrace{RR}_{S_0}underbrace{NR}_{S_1})=0.7cdot 0.7+0.3cdot 0.4=0.61.$$
                          which is:
                          $$begin{bmatrix}
                          underbrace{1}_{S_0} & underbrace{0}_{S_1} & underbrace{0}_{S_2} & underbrace{0}_{S_3}
                          end{bmatrix}
                          begin{bmatrix}
                          0.7 & 0 & 0.3 & 0 \
                          0.5 & 0 & 0.5 & 0 \
                          0 & 0.4 & 0 & 0.6 \
                          0 & 0.2 & 0 & 0.8 \
                          end{bmatrix}^2=
                          begin{bmatrix}
                          underbrace{0.49}_{S_0} & underbrace{0.12}_{S_1} & underbrace{0.21}_{S_2} & underbrace{0.18}_{S_3}end{bmatrix} Rightarrow\
                          mathbb P(S_0)+mathbb P(S_1)=0.49+0.12=0.61.$$
                          3) Probability of raining on Friday given that it rained on Monday and Tuesday. Can you figure this out both ways: probability tree diagram and markov chain formula? The initial state matrix is $(1,0,0,0)$, which must be multiplied by the cube of the transition matrix and the probabilities of $S_0$ and $S_1$ must be added.







                          share|cite|improve this answer












                          share|cite|improve this answer



                          share|cite|improve this answer










                          answered Jul 6 '18 at 6:06









                          farruhotafarruhota

                          20.2k2738




                          20.2k2738






























                              draft saved

                              draft discarded




















































                              Thanks for contributing an answer to Mathematics Stack Exchange!


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid



                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.


                              Use MathJax to format equations. MathJax reference.


                              To learn more, see our tips on writing great answers.




                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function () {
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f1467415%2ftransition-probability-matrix-of-a-markov-chain%23new-answer', 'question_page');
                              }
                              );

                              Post as a guest















                              Required, but never shown





















































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown

































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown







                              Popular posts from this blog

                              Ellipse (mathématiques)

                              Quarter-circle Tiles

                              Mont Emei