Markov Chain (Continuous Time)

Posted on by

1. Definition

t-step transition probability: Let P_{ij}(t) be the probability that the system is in state j in t time units, given the system is in state i now.

P_{ij}(t) = P(X(t+s) = j | X(s) = i)
                 = P(X(t) =j | X(0) = i)  (by stationarity)

2. Properties

Lemma 6.2 lim_{t to 0} frac{1-P_{ii}(h)}{h} = v_i

Lemma  6.2 b: lim_{h to 0} = frac{P_{ij}(h)}{h} = q_{ij} = v_i p_{ij}

Lemma 6.3: 

3. Forward Chapman-Kolmogorov Equations

P'_{ij}(t) = sum_{k neq j} q_{kj} P_{ik}(t) - v_j P_{ij}(t)

Proof:



Define q_{jj} = -v_j



Leave a Reply