Lecture 19: Joint, Conditional, and Marginal Distributions

Joint, Conditional, Marginal Dist.

• joint CDF

  • $F(x, y) = P(X \le x, Y \le y)$

• joint PDF

  • continuous case:

    $f(x, y) = \displaystyle \frac{\partial}{\partial x \partial y} F(x, y)$

    $P((x, y) \in A) = \iint_{A} f(x, y) dx dy$ in the any region of A

    cf. $\iint_{-∞}^{∞} f(x, y) dx dy = 1$

• marginal PDF of $X$

  • $\int_{-∞}^{∞} f(x, y) dy$

• conditional PDF of $Y|X$ is

  • $f_{Y|X} (y|x) = \displaystyle \frac{f_{X, Y}(x, y)}{f_X(x)} = \frac{f_{X|Y}(x|y) f_{Y}(y)}{f_{X}(x)}$ (Bayes rule)

• $X$, $Y$ indep. if

  • $f_{X, Y}(x, y) = f_{X}(x) f_{Y}(y)$, $f(x, y) = \begin{cases} \displaystyle \frac{1}{\pi}, {x^2 + y^2 \le 1 → y^2 \le 1-x^2} \\ 0, {otherwise} \\ \end{cases}$

  • ex) Unif in disc $x^2 + y^2 \le 1$

    

    • $f_{X}(x) = \displaystyle \int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}} \displaystyle \frac{2}{\pi} \sqrt{1-x^2}$, $-1 \le x \le 1$

    • $f_{Y|X}(y|x) = \displaystyle \frac{\frac{1}{\pi}}{\frac{2}{\pi} \sqrt{1-x^2}}$ if $-\sqrt{1-x^2} \le y \le \sqrt{1-x^2}$

    • $Y|X \sim Unif(-\sqrt{1-x^2}, \sqrt{1-x^2})$
      [which means $Y|X=x \sim Unif(-\sqrt{1-x^2}, \sqrt{1-x^2})$ ]

    • $f_{X, Y}(x, y) \ne f_{X}(x) f_{Y}(y)$
      so not independent.

2-D LOTUS

• Let ($X$, $Y$) have joint PDF $f(x, y)$, and let $g(x, y)$ be a real valued fn of $x$, $y$.

  • Then $E(g(X, Y)) = \int_{-∞}^{∞} \int_{-∞}^{∞} g(x, y) f(x, y) dx dy$

• Thm: If $X$, $Y$ are indep. them $E(XY) = E(X)E(Y)$

  -"Indep. implies uncorrelated"

• Proof(continuous case)

  • $E(XY) = \int_{-∞}^{∞} \int_{-∞}^{∞} xy f_X(x) f_Y(y) dx dy$

    $\int_{-∞}^{∞} yf_{Y}(y) \int_{-∞}^{∞} xf_X(x) dx dy = (EX)(EY)$

• Example: $X$, $Y$ i.i.d $\sim Unif(0, 1)$, find $E|X-Y|$

  • LOTUS: $\int_{0}^{1} \int_{0}^{1} |x-y| dx dy$

    $= \iint_{x>y} (x-y) dxdy + \iint_{x \le y} (y-x) dxdy = 2 \int_{0}^{1} \int_{y}^{1} (x-y) dxdy$

    $= 2 \int_{0}^{1} (\displaystyle \frac{x^2}{2} - yx \biggr|_{1}^{y}) dy = \frac{1}{3}$

• Let $M = max(X, Y)$, $L = min(X, Y)$

  • $|X-Y| = M-L$, $E(M-L) = \displaystyle \frac{1}{3} = E(M)-E(L) = \frac{1}{3}$

  • $E(M+L) = E(X+Y) = E(M) + E(L) = 1$

    $\Rightarrow E(M) = \displaystyle \frac{2}{3}$, $E(L) = \displaystyle \frac{1}{3}$

• Example: Chicken-egg problem

  • $N \sim Pois(\lambda)$ eggs, each hatches with prob. $p$ indep.

    • Let $X$ = # that hatch, so $X|N \sim Bin(N, p)$
    • Let $Y$ = # don't hatch, so $X+Y = N$
    • Find joint PMF of $X$, $Y$.

  • $P(X=i, Y=j) = \displaystyle \sum_{n=0}^{∞} P(X=i, Y=j | N=n) P(N=n)$

    $= P(X=i, Y=j | N=i+j) P(N=i+j)$

    $= P(X=i | N=i+j) P(N=i+j)$ → it's just Binomial! (x+y = n)

    cf. $P(X=3, Y=5 | N=10) = 0$, $P(X=3, Y=5 | N=2) = 0$

  • $P(X=i | N=i+j) P(N=i+j)$

    $= \displaystyle \frac{(i+j)!}{i! j!} p^{i} q^{j} \frac{e^{-\lambda} \lambda^{i+j}}{(i+j)!}$, $p+q = 1$

    $= (\displaystyle e^{-\lambda p}\frac{(\lambda p)^i}{i!}) (\displaystyle e^{-\lambda q}\frac{(\lambda q)^j}{j!})$

    $\Rightarrow X, Y$ are indep., $X \sim Pois(\lambda p)$, $Y \sim Pois(\lambda q)$

  • 푸아송 분포에 대한 무작위적인 확률은 독립성을 갖게 만든다.

• 출처 : Statistics 110, boostcourse