Lecture 4: Conditional Probability

Matching(continued)

• $A_j$ : jth card in desk is labeled; Find $P(\displaystyle ⋃^{n}_{j=1} A_j)$

• $P(A_1 \cup A_2 \cup\; ... \; \cup A_n)$, there are $\begin{pmatrix}n\\k\\ \end{pmatrix} = \frac{n!}{(n-k)!(k!)}$
  such terms, all same by symmetry.

• $P(\displaystyle ⋃^{n}_{j=1} A_j)$

  $= n \times \frac{1}{n} - \frac{(n-2)!}{n!} \times \frac{1}{n(n-1)} + \frac{n(n-1)(n-2)}{3} \times \frac{1}{n(n-1)(n-2)}-...$

  $= 1 - \frac{1}{2!} + \frac{1}{3!} - ... + (-1)^{n+1} \frac{1}{n!}$ -> Tayler's series

• so, $P(no match) = P(\displaystyle ⋃^{n}_{j=1} A_j^c)$

  $= 1 - 1 + \frac{1}{2!} - \frac{1}{3!} - ... + (-1)^n \frac{1}{n!}$

  $\approx \frac{1}{e} = 0.37$

Definition

• Events A, B are independent if $P(A \cap B) = P(A) P(B)$

• Note : Completely different from disjointness

• A, B, C are indep. if $P(A, B) = P(A) P(B)$, $P(A, C) = P(A) P(C)$, $P(B, C) = P(B) P(C)$, $P(A, B, C) = P(A) P(B) P(C)$

• Similarly for events $A_1, \; ... \; A_n$ : "indep. means multiply"

Newton-Pepys Problem (1693)

• Have fair dice; which is most likely?

  • (A) at least one 6 with 6 dice ← truth
    (B) ... two 6's ... 12 dice
    (C) ... three 6's ... 18 dice ← Pepys

  • $P(A) = 1 - (\frac{5}{6})^6 = 0.665$

    $P(B) = 1 - (\frac{5}{6})^12 - 12(\frac{1}{6})(\frac{5}{6})^{11} = 0.615$

    $P(B) = 1 - \displaystyle \sum_{k=0}^{2} \begin{pmatrix}18\\k\\ \end{pmatrix}(\frac{1}{6})^{k}(\frac{5}{6})^{18-k} = 0.597$ : Binomial Prob

    ex. | 6 | 6 | ... | not 6 | not 6 |

Conditional probabilities

• How should you update prob/beliefs/uncertainty based on new evidence?

  • "Conditioning is the soul of statistics."

• Definition $P(A|B) = \frac{P(A \cap B)}{P(B)}$, if $P(B) >0$ (A occurs when if B occurs)

  • Intuition 1: pebble word

    ex. 9 pebbles in S, total mass is 1.
    | o o x |
    | o x x |
    | x x x |

    • $P(A|B)$ : get rid of pebbles in $B^c$, renormalize to make mass 1 again

    • $P(A \cap B)$을 선택하면, 전체 집합인 $B$의 total mass가 더이상 1이 아니게 되므로 $P(B)$를 곱함으로써 renormalize 하는 것이다.

  • Intuition 2: frequentist world; repeat, except many times

    100101101
    001001011
    111111111
    ...

    • Bold repeats where B occured, among those, what fraction of time did A also occur?

Theorem

• Thrm 1:

  $P(A \cap B) = P(B)P(A|B) = P(A)P(B|A)$

• Thrm 2: [n! times] [Chain rule]

  $P(A_1, \;, ... \;, A_n) = P(A_1)P(A_2|A_1)P(A_3|A_1, A_2) ... P(A_n|A_1, A_2, ... ,A_{n-1})$

• Thrm 3: [Bayse's rule]

  $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

• 출처 : Statistics 110, boostcourse