Lecture 20: Multinomial and Cauchy

Continue to Lecture 19

• Distance between 2 i.i.d. variables

  • Ex) Find $E|Z_1 - Z_2|$, with $Z_1$, $Z_2$ $\sim i.i.d. N(0, 1)$

    • $Z_1 - Z_2$ : i.i.d. standard normal

  • Thm) $X \sim N(\mu_1, \sigma_1^{2})$, $Y \sim N(\mu_2, \sigma_2^{2})$, indep. then,

    • $X+Y \sim N(\mu_1+\mu_2, \sigma_1^{2}+\sigma_2^{2})$

  • Proof) Use MGFs: MGF of $X+Y$ is

    • $e^{\displaystyle \mu_1t + \frac{1}{2} \sigma_1^{2} t^{2}} e^{\displaystyle \mu_2t + \displaystyle \frac{1}{2} \sigma_2^{2} t^{2}} = e^{\displaystyle (\mu_1+\mu_2)t + \displaystyle \frac{1}{2} (\sigma_1^{2}+\sigma_2^{2}) t^{2}}$

    : MGF of $N(\mu_1+\mu_2, \sigma_1^{2}+\sigma_2^{2})$

  • Note $Z_1-Z_2 \sim N(0, 2)$, $2$ is just scale(척도).

    • $E|Z_1 - Z_2| = E|\sqrt{2} Z|$, $Z \sim N(0, 2)$ : It's just 1D LOTUS

      $= \sqrt{2}E|Z|$ = $\sqrt{2} \int_{-∞}^{∞} |z| \displaystyle \frac{1}{\sqrt{2 \pi}} e^{-z^2/2} dz = \displaystyle \frac{1}{\sqrt{2 \pi}}$ (even function)

Multinomial

• Defn/Story of $Mult(n, \vec{p})$

  • $\vec{p} = (p_1, ... p_k)$ prob. vector $p_j \ge 0$, $\sum_{j} p_j = 1$

    • $k$ parameters of prob. for one case!

  • $\vec{X} \sim Mult(n, \vec{p})$, $\vec{X} = (X_1, ... X_k)$

    • if have n objects, indep. putting into k categories, $p_j$ = $P(categoty \; j)$,

    • $X_j$ = # objects in catagory $j$

• Joint PMF $P(X_1 = n_1, ... X_n = n_k) = \displaystyle \frac{n!}{n_1! n_2! ... n_k!} p_1^{n_1} p_2^{n_2} ... p_k^{n_k}$

  • if $n_1 + ... +n_k = 1$ (0 otherwise)

• $\vec{X} \sim Mult_k(n, \vec{p})$, Find a Marginal dist.(주변 분포) of $X_j$

  • Then, $X_j \sim Bin(n, p_j)$

  • $E(X_j) = np_j$, $Var(X_j) = np_j(1-p_j)$

Lumping Property(일괄 처리)

• $\vec{X} = (X_1, X_2, ... X_10) \sim Mult(n, (p_1, ..., p_10))$

  • Let $\vec{Y} = (X_1, X_2, X_3 + ... +X_10)$,

    • Then $\vec{Y} \sim Mult(n, (p_1, p_2, p_3 + ... + p_10)$

Conditional Probability

• $\vec{X} \sim Mult(n, \vec{p})$, Then given $X_1 = n_1$

  • $(X_2, ... X_k) \sim Mult_{k-1} (n-n_1, (p_2', ... p_k'))$

  • with $p_2' = P(being \; in \; category \; 2 \; | \; not \; in \; category \; 1)$

    $= \displaystyle \frac{p_2}{1-p_1} + \frac{p_2}{p_2 + ... + p_k}$, $p_j' = \displaystyle \frac{p_j}{p_2 + ... + p_k}$

Cauchy interview Problem

• The Cauchy is dist. of ratio $T = \displaystyle \frac{X}{Y}$ with $X$, $Y$ i.i.d. $\sim N(0, 1)$.

  • Find PDF of T. (This is evel!)

• $P(\displaystyle \frac{X}{Y} \le t) = P(\displaystyle \frac{X}{|Y|} \le t)$ (Symmetry of $N(0, 1)$)

  $F(t) = P(X \le t|Y|) = \displaystyle \frac{1}{\sqrt{2 \pi}} \int_{-∞}^{∞} e^{-y^2/2} \int_{-∞}^{t|y|} \displaystyle \frac{1}{\sqrt{2 \pi}} e^{-x^2/2} dx dy$

  $= \displaystyle \frac{1}{\sqrt{2 \pi}} \int_{-∞}^{∞} e^{-y^2/2} \Phi (t|y|) dy = 2 • \displaystyle \sqrt{\frac{1}{2 \pi}} \int_{0}^{∞} e^{-y^2/2} \Phi (t|y|) dy$ (even function)

  • Then, what you gonna do next? → Find PDf means you get the above is CDF.
    • So, let's derivative of CDF ⇒ There is PDF!
  • If there is "well-behaved" function, sㅁhoud derivative first before integral

• PDF: $F'(t) = \displaystyle \sqrt{\frac{2}{\pi}} \int_{0}^{∞} ye^{-y^2/2} \displaystyle \sqrt{\frac{1}{2 \pi}} e^{-t^2y^2/2} dy = \frac{1}{\pi} \int_{0}^{∞} y e^{-(1+t^2)^2 y^2/2} dy$

  • Let's say $u = (1+t^2)y^2/2$, $du = (1+t^2) ydy$

  $= \displaystyle \frac{1}{\pi (1+t^2)}$

• $P(X \le t|Y|) = \int_{-∞}^{∞} P(X \le t|Y| | Y=y) \phi (y) dy$

  • $\phi$ is $N(0, 1)$ PDF

  $= \int_{-∞}^{∞} \Phi (t|y|) \phi(y) dy$

• 출처 : Statistics 110, boostcourse