## 1. Definition

** t-step transition probability**: Let be the probability that the system is in state j in t time units, given the system is in state i now.

= (by stationarity)

## 2. Properties

__Lemma 6.2__

** Lemma 6.2 b**:

**Lemma 6.3: **

## 3. Forward Chapman-Kolmogorov Equations

__Define __