Chapman Kolmogorov Equations for Continuous Time Mc

In mathematics and statistics, in the context of Markov processes, the Kolmogorov equations, including Kolmogorov forward equations and Kolmogorov backward equations, are a pair of systems of differential equations that describe the time evolution of the process's distribution. This article, as opposed to the article titled Kolmogorov equations, focuses on the scenario where we have a continuous-time Markov chain (so the state space ${\displaystyle \Omega }$ is countable). In this case, we can treat the Kolmogorov equations as a way to describe the probability ${\displaystyle P(x,s;y,t)}$ , where ${\displaystyle x,y\in \Omega }$ (the state space) and ${\displaystyle t>s,t,s\in \mathbb {R} _{\geq 0}}$ are the final and initial times, respectively.

The equations [edit]

For the case of a countable state space we put ${\displaystyle i,j}$ in place of ${\displaystyle x,y}$ . The Kolmogorov forward equations read

${\displaystyle {\frac {\partial P_{ij}}{\partial t}}(s;t)=\sum _{k}P_{ik}(s;t)A_{kj}(t)}$ ,

where ${\displaystyle A(t)}$ is the transition rate matrix (also known as the generator matrix),

while the Kolmogorov backward equations are

${\displaystyle {\frac {\partial P_{ij}}{\partial s}}(s;t)=-\sum _{k}A_{ik}(s)P_{kj}(s;t)}$

The functions ${\displaystyle P_{ij}(s;t)}$ are continuous and differentiable in both time arguments. They represent the probability that the system that was in state ${\displaystyle i}$ at time ${\displaystyle s}$ jumps to state ${\displaystyle j}$ at some later time ${\displaystyle t>s}$ . The continuous quantities ${\displaystyle A_{ij}(t)}$ satisfy

${\displaystyle A_{ij}(t)=\left[{\frac {\partial P_{ij}}{\partial u}}(t;u)\right]_{u=t},\quad A_{jk}(t)\geq 0,\ j\neq k,\quad \sum _{k}A_{jk}(t)=0.}$

Background [edit]

The original derivation of the equations by Kolmogorov starts with the Chapman–Kolmogorov equation (Kolmogorov called it fundamental equation) for time-continuous and differentiable Markov processes on a finite, discrete state space.^[1] In this formulation, it is assumed that the probabilities ${\displaystyle P(i,s;j,t)}$ are continuous and differentiable functions of ${\displaystyle t>s}$ . Also, adequate limit properties for the derivatives are assumed. Feller derives the equations under slightly different conditions, starting with the concept of purely discontinuous Markov process and then formulating them for more general state spaces.^[2] Feller proves the existence of solutions of probabilistic character to the Kolmogorov forward equations and Kolmogorov backward equations under natural conditions.^[2]

Relation with the generating function [edit]

Still in the discrete state case, letting ${\displaystyle s=0}$ and assuming that the system initially is found in state ${\displaystyle i}$ , the Kolmogorov forward equations describe an initial-value problem for finding the probabilities of the process, given the quantities ${\displaystyle A_{jk}(t)}$ . We write ${\displaystyle p_{k}(t)=P_{ik}(0;t)}$ where ${\displaystyle \sum _{k}p_{k}(t)=1}$ , then

${\displaystyle {\frac {dp_{k}}{dt}}(t)=\sum _{j}A_{jk}(t)p_{j}(t);\quad p_{k}(0)=\delta _{ik},\qquad k=0,1,\dots .}$

For the case of a pure death process with constant rates the only nonzero coefficients are ${\displaystyle A_{j,j-1}=\mu j,\ j\geq 1}$ . Letting

${\displaystyle \Psi (x,t)=\sum _{k}x^{k}p_{k}(t),\quad }$

the system of equations can in this case be recast as a partial differential equation for ${\displaystyle {\Psi }(x,t)}$ with initial condition ${\displaystyle \Psi (x,0)=x^{i}}$ . After some manipulations, the system of equations reads,^[3]

${\displaystyle {\frac {\partial \Psi }{\partial t}}(x,t)=\mu (1-x){\frac {\partial {\Psi }}{\partial x}}(x,t);\qquad \Psi (x,0)=x^{i},\quad \Psi (1,t)=1.}$

History [edit]

A brief historical note can be found at Kolmogorov equations.

See also [edit]

• Kolmogorov equations
• Master equation (in physics and chemistry), a synonym of "Kolmogorov equations" for many continuous-time Markov chains appearing in physics and chemistry.

References [edit]

1. ^ Kolmogoroff, A. (1931). "Über die analytischen Methoden in der Wahrscheinlichkeitsrechnung". Mathematische Annalen. 104: 415–458. doi:10.1007/BF01457949.
2. ^ ^{a b} Feller, Willy (1940) "On the Integro-Differential Equations of Purely Discontinuous Markoff Processes", Transactions of the American Mathematical Society, 48 (3), 488-515 JSTOR 1990095
3. ^ Bailey, Norman T.J. (1990) The Elements of Stochastic Processes with Applications to the Natural Sciences, Wiley. ISBN 0-471-52368-2 (page 90)

lancasterduchich.blogspot.com

Source: https://en.wikipedia.org/wiki/Kolmogorov_equations_(continuous-time_Markov_chains)

Share this post