선형대수학 기본

Projection of $x$ onto $y$

Some properties of determinant

Def)
Let $A \in \mathbb{R}^{k \times k}$
$|A|=det(A)= \left\{\begin{matrix} a_{11} & k=1\\ \sum_{j=1}^{k} a_{1j} |M_{1j}| (-1)^{1+j} & k>1 \end{matrix}\right.$
where $M_{1j}:(k-1) \times (k-1) \space matrix$ obtained by deleting 1st row and jth column of $A$

Properties)
① $|cA|=c^k |A|$
② $|A^T|=|A|$
③ $|AB|=|A| |B|=|BA|$
④ $|A^{-1}|=1/|A|$ (If $A$ is invertible)

✔︎ For any $A \in \mathbb{R}^{p \times q}$ and $B \in \mathbb{R}^{q \times p}$, $-|I_p + AB|=|I_q + BA|$

동치 명제

For $A \in \mathbb{R}^{p \times p}$
$|A| \neq 0$ <=> Columns of $A$ are linearly indep.
<=> $A$ is invertible <=> $rank(A)=p$

Some theorems related to positive definite

Theorem1)
If $A \in \mathbb{R}^{p \times p}$ is positive definite and $B \in \mathbb{R}^{p \times q}$ has a full rank $q(\leq p)$, $B^T AB$ is p.d.
pf)

Theorem2)
If $A \in \mathbb{R}^{p \times p}$ is p.d., $A^{-1}$ is p.d.
pf)

Eigenvalues and eigenvectors

Def)
For a matrix $A \in \mathbb{R}^{p \times p}$,
if $Ax=\lambda x$, $\left\{\begin{matrix} \lambda:eigenvalue \\ x:eigenvector \end{matrix}\right.$

Remarks)
When $A$ is symmetric,
① $A$ has $p$ real eigenvalues
② If $\lambda_i$ and $\lambda_j$ are its distinct eigenvalues, then $x_i \perp x_j$
pf)
Note that $(Ax_i)^T x_j=(\lambda_i x_i)^T x_j=\lambda_i x_i^T x_j$
and $(Ax_i)^T x_j=x_i^T A x_j=x_i^T(\lambda_j x_j)=\lambda_j x_i^T x_j$
Thus we have $\lambda_i x_i^T x_j=\lambda_j x_i^T x_j$
Since $\lambda_i \neq \lambda_j$, $x_i^T x_j=0$

Orthogonal matrix

Def)
A matrix $P \in \mathbb{R}^{p \times p}$ is said to be orthogonal if $P^T P=I_p$ i.e. $P^T=P^{-1}$

Remarks)
Let $P \in \mathbb{R}^{p \times p}$ is orthogonal
① $P$ preserves distance any $x \in \mathbb{R}^p$ and $y \in \mathbb{R}^p$
$\because \left\| Px-Py \right\|^2=(x-y)^T P^T P (x-y) = \left\| x-y \right\|^2$
② If $|P|= \pm 1$
$\because |I_p|=1=|P^T P|=|P|^2$ So, $|P| = \pm1$

Idempotent matrix

Def)
A matrix $A \in \mathbb{R}^{p \times p}$ is idempotent if $A^2=A$

Remarks)
① If $A \in \mathbb{R}^{p \times p}$ is idempotent, its eigenvalues are either 0 or 1
$\because \lambda x=Ax=A^2 x=\lambda Ax=\lambda^2 x$ So, $\lambda=0 \space or \space 1$