Random vectors and matrices

Suppose $X$ and $Y$ are vector
covariance matrix 설명

• Linearity and bilinearity

  • $E(aX+bY)=aE(X)+bE(Y),\space \forall a,b \in \mathbb{R}$
  • $Cov(X+Y,Z)=Cov(X,Z)+Cov(Y,Z)$
  • $Cov(X,Y+Z)=Cov(X,Y)+Cov(X,Z)$
  • For any matrices $A$ and $B$, $\left\{\begin{matrix} E(AX)=AE(X)\\ E(XB)=E(X)B\\ Cov(AX,Y)=ACov(X,Y)\\ Cov(X,BY)=Cov(X,Y)B^T \end{matrix}\right.$

• Variance matrix

  • $Var(AX)=A Var(X) A^T=Cov(AX,AX)$
  • $Var(X+b)=Var(X)$
  • $Var(X+Y)=Var(X)+Cov(X,Y)+Cov(Y,X)+Var(Y) \space (Cov^T(X,Y)=Cov(Y,X))$
  • $Var(X)$ is singular <=> $\exists a(\neq0) \in \mathbb{R}^p$ s.t. $P(a^T(X-E(X))=0)=1$
  • Correlation matrix
    $Corr(X)=\frac{Cov(X_i,X_j)}{\sqrt{Var(X_i)} \sqrt{Var(X_j)}}=diag(\frac{1}{\sqrt{Var(X_i)}}) Var(X) diag(\frac{1}{\sqrt{Var(X_j)}})$

• Canonical Correlation
  For $X_1 \in \mathbb{R}^k$ and $X_2 \in \mathbb{R}^m$, we have $\underset{a,b,c,d}{max} \space Corr(c^T X_2+d,a^T X_1+d )=\sqrt{\underset{1 \leq i \leq m}{max} \lambda_i}$
  where $\lambda_1, \cdots, \lambda_m$ are eigenvalues of $\Sigma_{22}^{-1/2} \Sigma_{21} \Sigma_{11}^{-1} \Sigma_{12} \Sigma_{22}^{-1/2}$
  pf)