Basic ideas of the Principal Component Analysis (PCA)

The principal component analysis deals with the problem of fitting a low-dimensional affine subspace $S$ of dimension $d$ much smaller than the total dimension $D$ of the problem at hand (our data set). Mathematically it can be formulated as a statistical problem or a geometric problem. In our discussion of the theorem for the classical PCA, we will stay with a statistical approach. Historically, the PCA was first formulated in a statistical setting in order to estimate the principal component of a multivariate random variable.

We have a data set defined by a design/feature matrix $\boldsymbol{X}$ (see below for its definition)

• Each data point is determined by $p$ extrinsic (measurement) variables
• We may want to ask the following question: Are there fewer intrinsic variables (say $d < < p$) that still approximately describe the data?
• If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do.

• «
• 1
• ...
• 20
• 21
• 22
• 23
• 24
• 25
• 26
• 27
• 28
• 29
• 30
• 31
• 32
• 33
• 34
• 35
• 36
• 37
• ...
• 57
• »