Markov processes, the probabilities

Since both \( W \) and \( w \) represent probabilities, they have to be normalized, i.e., we require that at each time step we have $$ \begin{equation*} \sum_i w_i(t) = 1, \end{equation*} $$ and $$ \begin{equation*} \sum_j W(j\rightarrow i) = 1, \end{equation*} $$ which applies for all \( j \)-values. The further constraints are \( 0 \le W_{ij} \le 1 \) and \( 0 \le w_{j} \le 1 \). Note that the probability for remaining at the same place is in general not necessarily equal zero.