If we consider the above covariance as a matrix C_{ij}=\mathrm{cov}(X_i,\,X_j) , then the diagonal elements are just the familiar variances, C_{ii} = \mathrm{cov}(X_i,\,X_i) = \mathrm{var}(X_i) . It turns out that all the off-diagonal elements are zero if the stochastic variables are uncorrelated. This is easy to show, keeping in mind the linearity of the expectation value. Consider the stochastic variables X_i and X_j , ( i\neq j ):
\begin{align} \mathrm{cov}(X_i,\,X_j) &= \langle(x_i-\langle x_i\rangle)(x_j-\langle x_j\rangle)\rangle \tag{6}\\ &=\langle x_i x_j - x_i\langle x_j\rangle - \langle x_i\rangle x_j + \langle x_i\rangle\langle x_j\rangle\rangle \tag{7}\\ &=\langle x_i x_j\rangle - \langle x_i\langle x_j\rangle\rangle - \langle \langle x_i\rangle x_j\rangle + \langle \langle x_i\rangle\langle x_j\rangle\rangle \tag{8}\\ &=\langle x_i x_j\rangle - \langle x_i\rangle\langle x_j\rangle - \langle x_i\rangle\langle x_j\rangle + \langle x_i\rangle\langle x_j\rangle \tag{9}\\ &=\langle x_i x_j\rangle - \langle x_i\rangle\langle x_j\rangle \tag{10} \end{align}