Spectral Decomposition

Let $A \in \mathbb{R}^{p \times p}$ be a symmetric matrix with eigenvalues $\lambda_1 \geq \cdots \geq \lambda_p$ and eigenvectors $\xi_1 \geq \cdots \geq \xi_p (\in \mathbb{R}^p)$
Then for $P=(\xi_1,\cdots,\xi_p) \in \mathbb{R}^{p \times p}$ and $\Lambda=diag(\lambda_j) \in \mathbb{R}^{p \times p}$
$A=\sum_{i=1}^{p} \lambda_i \xi_i \xi_i^T=P \Lambda P^T$
pf)
Note that $A \xi_i=\lambda_i \xi_i \space (i=1,\cdots,p)$
Since $P$ is orthogonal, $A=P \Lambda P^T$

• Remarks

  Let $A \in \mathbb{R}^{p \times p}$ be symmetric with eigenvalues $\lambda_1, \cdots, \lambda_p$
  
  ① $A^{-1}$ has eigenvalues $1/\lambda_1, \cdots, 1/\lambda_p$ (eigenvector는 동일)
  $\because Ax=\lambda x => x=\lambda A^{-1}x => \lambda^{-1}x=A^{-1}x$
  
  ② $tr(A)=\sum_{i=1}^{p} \lambda_i$ and $|A|=\prod_{i=1}^p \lambda_i$
  $\because$ Using spectral decomposition, $tr(A)=tr(P \Lambda P^T)=tr(\Lambda)$ and $|A|=|P \Lambda P^T|=|P| |\Lambda| |P^{-1}|=|\Lambda|=\prod_{i=1}^p \lambda_i$
  
  ③ $rank(A)$ is the number of nonzero eigenvalues of $A$
  $\because$ Let $k$ be the number of nonzero eigenvalues of $A$
  
  $+$ Let $A$ be positive definite
  
  ④ Eigenvalues of $A$ are positive
  $\because$ By spectral decomposition, we have $x^T A x=x^T P \Lambda P^T x=\sum_{j=1}^{p} \lambda_j (\xi_j^T x)^2$
  If $\lambda_{j^*} \leq 0$ for some $j^*$, $x=\xi_{j^*}$ gives $x^T Ax=\lambda_{j^*} \leq 0$ => contradiction
  
  ⑤ $A^{1/2}$ is given by $A^{1/2}=P \Lambda^{1/2} P^T$ where $\Lambda^{1/2}=diag(\sqrt\lambda_1, \cdots, \sqrt \lambda_p)$

Theorem)
Suppose $A \in \mathbb{R}^{p \times p}$ is nonnegative definite with eigenvalues $\lambda_1 \geq \cdots \geq \lambda_p \geq 0$
Then, for $x \neq 0$, $\lambda_p \leq \frac{x^T Ax}{x^T x} \leq \lambda_1$
pf)