Markov processes

For the Markov process we have a transition probability from a position \( x=jl \) to a position \( x=il \) given by $$ \begin{equation*} W_{ij}(\epsilon)=W(il-jl,\epsilon)=\left\{\begin{array}{cc}\frac{1}{2} & |i-j| = 1\\ 0 & \mathrm{else} \end{array} \right. , \end{equation*} $$ where \( W_{ij} \) is normally called the transition probability and we can represent it, see below, as a matrix. Here we have specialized to a case where the transition probability is known.

Our new PDF \( w_i(t=\epsilon) \) is now related to the PDF at \( t=0 \) through the relation $$ \begin{equation*} w_i(t=\epsilon) =\sum_{j} W(j\rightarrow i)w_j(t=0). \end{equation*} $$

This equation represents the discretized time-development of an original PDF with equal probability of jumping left or right.