This means the vectors \( \boldsymbol{v}_i \) of the orthogonal matrix \( \boldsymbol{V} \) are the eigenvectors of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) with eigenvalues given by the singular values squared, that is
$$ \left(\boldsymbol{X}^T\boldsymbol{X}\right)\boldsymbol{v}_i=\boldsymbol{v}_i\sigma_i^2. $$In other words, each non-zero singular value of \( \boldsymbol{X} \) is a positive square root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). It means also that the columns of \( \boldsymbol{V} \) are the eigenvectors of \( \boldsymbol{X}^T\boldsymbol{X} \). Since we have ordered the singular values of \( \boldsymbol{X} \) in a descending order, it means that the column vectors \( \boldsymbol{v}_i \) are hierarchically ordered by how much correlation they encode from the columns of \( \boldsymbol{X} \).
Note that these are also the eigenvectors and eigenvalues of the Hessian matrix.