Covariance

Def) Covariance of $X$ & $Y$
$Cov(X,Y)=E[(X-E(X))(Y-E(Y))]$

Properties)
① $Cov(X,Y) = E(XY)-E(X)E(Y)$
② $Cov(X,Y) = Cov(Y,X)$
③ $Cov(X,X) = Var(X)$
④ $Cov(aX+b, cY+d) = ac Cov(X,Y)$
⑤ $Cov(aX+bY+c, dX+eY+f) = adVar(X) + aeCov(X,Y) + bdCov(X,Y) + beVar(Y)$

✔︎ If $X,Y$ are indep., $Cov(X,Y)$ = 0
(역은 $X,Y$가 정규분포를 따를 때만 성립)
✔︎ Covariance는 Variance에 의존하기 때문에 두 변수 간 선형관계를 판단할 수 없다.

Correlation Coefficient

Def) Correlation Coefficient of $X,Y$
$\rho_{X,Y} = \frac{Cov(X,Y)}{\sqrt{Var(X)} \sqrt{Var(Y)}}$

Properties)
① $Corr(X,X) = 1$
② $Corr(aX+b, cY+d) = sign(ac)Corr(X,Y)$
$sign(ac) = \left\{\begin{matrix} 1 & ac>0\\ -1 & ac<0 \end{matrix}\right.$
③ $-1 \leq Corr(X,Y) \leq 1$

Variance - Covariance Matrix

Def)
$\sum=Cov(\underline{X})=E[(\underline{X}-E(\underline{X}))(\underline{X}-E(\underline{X}))^T]=\begin{bmatrix} Var(X_1)& Cov(X_1,X_2) & \cdots & Cov(X_1,X_n) \\ Cov(X_2,X_1)& Var(X_2) & \cdots & Cov(X_2, X_n)\\ \vdots& \vdots & \vdots & \vdots \\ Cov(X_n,X_1)& Cov(X_n,X_2)& \cdots & Var(X_n) \\ \end{bmatrix}$

Theorem)
Suppose $\underline{X}=(X_1,...,X_n)^T,\underline{A}=m \times n \space matrix, \underline{b}= m \times 1 \space vector$
① $E(\underline{A} \underline{X} + \underline{b}) = \underline{A}E(\underline{X}) + \underline{b}$
② $Cov(\underline{X}) = E(\underline{X} \underline{X}^T) - E(\underline{X})E^T(\underline{X})$
③ $Cov(\underline{A} \underline{X} + \underline{b}) = \underline{A} Cov(\underline{X}) \underline{A}^T$