PCA例题
已知样本集合的协方差矩阵为
C x = 1 10 [ 3 1 1 1 3 − 1 1 − 1 3 ] {\bm C}_x = \frac{1}{10} \begin{bmatrix} 3 & 1 & 1 \\ 1 & 3 & -1 \\ 1 & -1 & 3 \end{bmatrix} Cx=101 31113−11−13
使用PCA方法将样本向量降到二维 。
求解
计算 C x {\bm C}_x Cx的特征值得到：
λ 1 = 0.1 , λ 2 = λ 3 = 0.4 \lambda_1 = 0.1, \quad \lambda_2 = \lambda_3 = 0.4 λ1=0.1,λ2=λ3=0.4
其对应的特征向量分别为：
u 1 = 1 3 [ 1 − 1 − 1 ] , u 2 = 1 6 [ 2 1 1 ] , u 3 = 1 2 [ 0 1 − 1 ] {\bm u}_1 = \frac{1}{\sqrt{3}} \begin{bmatrix} 1 \\ -1 \\ -1 \end{bmatrix}, \quad {\bm u}_2 = \frac{1}{\sqrt{6}} \begin{bmatrix} 2 \\ 1 \\ 1 \end{bmatrix}, \quad {\bm u}_3 = \frac{1}{\sqrt{2}} \begin{bmatrix} 0 \\ 1 \\ -1 \end{bmatrix} u1=31 1−1−1 ,u2=61 211 ,u3=21 01−1
取最大的两个特征值 λ 2 , λ 3 \lambda_2, \lambda_3 λ2,λ3对应的特征向量构成压缩空间，形成新的向量：
y ~ = [ u 2 u 3 ] ⊤ x = [ 2 / 6 1 / 6 1 / 6 0 1 / 2 − 1 / 2 ] x {\tilde {\bm y}} = \begin{bmatrix} {\bm u}_2 & {\bm u}_3 \end{bmatrix}^\top {\bm x} = \begin{bmatrix} 2/\sqrt{6} & 1/\sqrt{6} & 1/\sqrt{6} \\ 0 & 1/\sqrt{2} & -1/\sqrt{2} \end{bmatrix} {\bm x} y~=[u2u3]⊤x=[2/601/61/21/6−1/2]x
K-L 变换为：
y = W ⊤ x = [ u 1 u 2 u 3 ] ⊤ x = [ 1 / 3 − 1 / 3 − 1 / 3 2 / 6 1 / 6 1 / 6 0 1 / 2 − 1 / 2 ] x {\bm y} = {\bm W}^\top {\bm x} = \begin{bmatrix} {\bm u}_1 & {\bm u}_2 & {\bm u}_3 \end{bmatrix}^\top {\bm x} = \begin{bmatrix} 1/\sqrt{3} & -1/\sqrt{3} & -1/\sqrt{3} \\ 2/\sqrt{6} & 1/\sqrt{6} & 1/\sqrt{6} \\ 0 & 1/\sqrt{2} & -1/\sqrt{2} \end{bmatrix} {\bm x} y=W⊤x=[u1u2u3]⊤x= 1/32/60−1/31/61/2−1/31/6−1/2 x