Explanation of Markov transition function
$begingroup$
Here the definition of my course (in the picture below). Could someone explain me the Chapman-Kolomogorov equation ? I don't really understand what it mean. Also, I tried to make a parallel with discrete Markov chain, I don't see the link between continuous and discrete Markov chain. Is the motivation behind the same ? (in discrete time $(X_n)$ is a Markov chain if $$mathbb P{X_{n+1}=xmid sigma (X_{k}mid kleq n)}=mathbb P{X_{n+1}=xmid X_n}.$$ Also, $P_{s,t}(x,dy)$ is the regular version of the conditional distribution of $X_t$ given $X_s$... I'm not really sure what it mean, is it $$P_{s,t}(x,dy)=mathbb P{X_tmid X_s} ?$$
But it doesn't really make sense.
markov-chains
asked Jan 1 at 17:24
NewMathNewMath
4059
$endgroup$
add a comment |
$begingroup$
Here the definition of my course (in the picture below). Could someone explain me the Chapman-Kolomogorov equation ? I don't really understand what it mean. Also, I tried to make a parallel with discrete Markov chain, I don't see the link between continuous and discrete Markov chain. Is the motivation behind the same ? (in discrete time $(X_n)$ is a Markov chain if $$mathbb P{X_{n+1}=xmid sigma (X_{k}mid kleq n)}=mathbb P{X_{n+1}=xmid X_n}.$$ Also, $P_{s,t}(x,dy)$ is the regular version of the conditional distribution of $X_t$ given $X_s$... I'm not really sure what it mean, is it $$P_{s,t}(x,dy)=mathbb P{X_tmid X_s} ?$$
But it doesn't really make sense.
markov-chains
asked Jan 1 at 17:24
NewMathNewMath
4059
$endgroup$
$begingroup$
why the downvote ? May be it's so obvious that someone downvoted, and thus, this person could answer my question and explain in what my question is so obvious, and I promiss that I'll erase my question ;) The reason of the downvoter it's that my question is not clear. How could I be more clear that : What mean $P_{s,u}(x,A)=int_E P_{s,t}(x,dy)P_{t,u}(y,A)$ ? And what is the relation with discrete Markov chain.
$endgroup$
– NewMath
Jan 1 at 18:01
add a comment |
$begingroup$
Here the definition of my course (in the picture below). Could someone explain me the Chapman-Kolomogorov equation ? I don't really understand what it mean. Also, I tried to make a parallel with discrete Markov chain, I don't see the link between continuous and discrete Markov chain. Is the motivation behind the same ? (in discrete time $(X_n)$ is a Markov chain if $$mathbb P{X_{n+1}=xmid sigma (X_{k}mid kleq n)}=mathbb P{X_{n+1}=xmid X_n}.$$ Also, $P_{s,t}(x,dy)$ is the regular version of the conditional distribution of $X_t$ given $X_s$... I'm not really sure what it mean, is it $$P_{s,t}(x,dy)=mathbb P{X_tmid X_s} ?$$
But it doesn't really make sense.
markov-chains
asked Jan 1 at 17:24
NewMathNewMath
4059
$endgroup$
Here the definition of my course (in the picture below). Could someone explain me the Chapman-Kolomogorov equation ? I don't really understand what it mean. Also, I tried to make a parallel with discrete Markov chain, I don't see the link between continuous and discrete Markov chain. Is the motivation behind the same ? (in discrete time $(X_n)$ is a Markov chain if $$mathbb P{X_{n+1}=xmid sigma (X_{k}mid kleq n)}=mathbb P{X_{n+1}=xmid X_n}.$$ Also, $P_{s,t}(x,dy)$ is the regular version of the conditional distribution of $X_t$ given $X_s$... I'm not really sure what it mean, is it $$P_{s,t}(x,dy)=mathbb P{X_tmid X_s} ?$$
But it doesn't really make sense.
markov-chains
markov-chains
asked Jan 1 at 17:24
NewMathNewMath
4059
asked Jan 1 at 17:24
NewMathNewMath
4059
edited Jan 1 at 17:44
NewMath
asked Jan 1 at 17:24
NewMathNewMath
4059
asked Jan 1 at 17:24
NewMathNewMath
4059
asked Jan 1 at 17:24
NewMathNewMath
4059
4059
$begingroup$
why the downvote ? May be it's so obvious that someone downvoted, and thus, this person could answer my question and explain in what my question is so obvious, and I promiss that I'll erase my question ;) The reason of the downvoter it's that my question is not clear. How could I be more clear that : What mean $P_{s,u}(x,A)=int_E P_{s,t}(x,dy)P_{t,u}(y,A)$ ? And what is the relation with discrete Markov chain.
$endgroup$
– NewMath
Jan 1 at 18:01
add a comment |
$begingroup$
why the downvote ? May be it's so obvious that someone downvoted, and thus, this person could answer my question and explain in what my question is so obvious, and I promiss that I'll erase my question ;) The reason of the downvoter it's that my question is not clear. How could I be more clear that : What mean $P_{s,u}(x,A)=int_E P_{s,t}(x,dy)P_{t,u}(y,A)$ ? And what is the relation with discrete Markov chain.
$endgroup$
– NewMath
Jan 1 at 18:01
$begingroup$
why the downvote ? May be it's so obvious that someone downvoted, and thus, this person could answer my question and explain in what my question is so obvious, and I promiss that I'll erase my question ;) The reason of the downvoter it's that my question is not clear. How could I be more clear that : What mean $P_{s,u}(x,A)=int_E P_{s,t}(x,dy)P_{t,u}(y,A)$ ? And what is the relation with discrete Markov chain.
$endgroup$
– NewMath
Jan 1 at 18:01
$begingroup$
why the downvote ? May be it's so obvious that someone downvoted, and thus, this person could answer my question and explain in what my question is so obvious, and I promiss that I'll erase my question ;) The reason of the downvoter it's that my question is not clear. How could I be more clear that : What mean $P_{s,u}(x,A)=int_E P_{s,t}(x,dy)P_{t,u}(y,A)$ ? And what is the relation with discrete Markov chain.
$endgroup$
– NewMath
Jan 1 at 18:01
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
You can think of a continuous-time Markov process as being fully characterized by both an "embedded" discrete-time Markov chain that governs the probabilities of transitions between states (call them $Q_{ij}$), as well as some holding time parameters $lambda_i$ that represent the average rates at which one transitions out of a state $i$. (Those holding times are always distributed exponentially, so knowing $lambda_i$ is enough to characterize their distribution.)
You can then construct an evolution equation, sometimes called a master equation, for the continuous-time transition probabilities $P_{ij}(t)$ using these parameters.
The Chapman-Kolmogorov equations for continuous-time Markov processes are "the same thing" they were in the discrete version: an identity for the transition probabilities based on conditioning on an intermediate step and exploiting the Markov property.
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3058674%2fexplanation-of-markov-transition-function%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
You can think of a continuous-time Markov process as being fully characterized by both an "embedded" discrete-time Markov chain that governs the probabilities of transitions between states (call them $Q_{ij}$), as well as some holding time parameters $lambda_i$ that represent the average rates at which one transitions out of a state $i$. (Those holding times are always distributed exponentially, so knowing $lambda_i$ is enough to characterize their distribution.)
You can then construct an evolution equation, sometimes called a master equation, for the continuous-time transition probabilities $P_{ij}(t)$ using these parameters.
The Chapman-Kolmogorov equations for continuous-time Markov processes are "the same thing" they were in the discrete version: an identity for the transition probabilities based on conditioning on an intermediate step and exploiting the Markov property.
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
add a comment |
$begingroup$
You can think of a continuous-time Markov process as being fully characterized by both an "embedded" discrete-time Markov chain that governs the probabilities of transitions between states (call them $Q_{ij}$), as well as some holding time parameters $lambda_i$ that represent the average rates at which one transitions out of a state $i$. (Those holding times are always distributed exponentially, so knowing $lambda_i$ is enough to characterize their distribution.)
You can then construct an evolution equation, sometimes called a master equation, for the continuous-time transition probabilities $P_{ij}(t)$ using these parameters.
The Chapman-Kolmogorov equations for continuous-time Markov processes are "the same thing" they were in the discrete version: an identity for the transition probabilities based on conditioning on an intermediate step and exploiting the Markov property.
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
add a comment |
$begingroup$
You can think of a continuous-time Markov process as being fully characterized by both an "embedded" discrete-time Markov chain that governs the probabilities of transitions between states (call them $Q_{ij}$), as well as some holding time parameters $lambda_i$ that represent the average rates at which one transitions out of a state $i$. (Those holding times are always distributed exponentially, so knowing $lambda_i$ is enough to characterize their distribution.)
You can then construct an evolution equation, sometimes called a master equation, for the continuous-time transition probabilities $P_{ij}(t)$ using these parameters.
The Chapman-Kolmogorov equations for continuous-time Markov processes are "the same thing" they were in the discrete version: an identity for the transition probabilities based on conditioning on an intermediate step and exploiting the Markov property.
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
You can think of a continuous-time Markov process as being fully characterized by both an "embedded" discrete-time Markov chain that governs the probabilities of transitions between states (call them $Q_{ij}$), as well as some holding time parameters $lambda_i$ that represent the average rates at which one transitions out of a state $i$. (Those holding times are always distributed exponentially, so knowing $lambda_i$ is enough to characterize their distribution.)
You can then construct an evolution equation, sometimes called a master equation, for the continuous-time transition probabilities $P_{ij}(t)$ using these parameters.
The Chapman-Kolmogorov equations for continuous-time Markov processes are "the same thing" they were in the discrete version: an identity for the transition probabilities based on conditioning on an intermediate step and exploiting the Markov property.
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
answered Jan 1 at 18:10
aghostinthefiguresaghostinthefigures
1,2641217
1,2641217
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3058674%2fexplanation-of-markov-transition-function%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
why the downvote ? May be it's so obvious that someone downvoted, and thus, this person could answer my question and explain in what my question is so obvious, and I promiss that I'll erase my question ;) The reason of the downvoter it's that my question is not clear. How could I be more clear that : What mean $P_{s,u}(x,A)=int_E P_{s,t}(x,dy)P_{t,u}(y,A)$ ? And what is the relation with discrete Markov chain.
$endgroup$
– NewMath
Jan 1 at 18:01