[ RV ] 02. Random Variables

Def: A "random variable" $X(u)$ is a mapping from the sample space to the real line;
In other words, an assignment of a number to every possible outcome.
$X$ → real #
$u$ → sample point

• Mathematically, a random variable is a function from the sample space $U$ to real numbers $R$
  $X(u)$ : $u$ → $R$
• We can have several random variables defined on the same sample space
• discrete RV / continuous RV
• Ex_ $X$ ($f_i$) = 10$i$, in die experiment
  X $\in$ { 10, 20, 30, 40, 50, 60 } ~ discrete RV
• Ex_ $U$=(0, 12], $X(u)$=$u^2$, $X$ $\in$ (0, 144] ~ continuous RV

Probability mass function ( PMF )

$P_X(x) = P[X = x]$
$X$ → RV
$x$ → integer

Ex_ $X(u)$ = { 10, 20, 30, 40, 50, 60 }
$P_X(10)$ = $P_X(20)$ = ... = $P_X(60)$ = 1/6

• used for discrete RVs
• $P_X(x)$ $\ge$ 0, $\Sigma_xP_X(x)$ = 1

Ex_ Binomial random variable
X ~ number of heads in n independent coin tosses
$P_X(x)$ = $(^n_x)p^x(1-p)^{n-x}$ ~ " Binomial PMF "

Ex_ Geometric random variable
X ~ number of coin tosses until the first head
$P_X(x)$ = $(1-p)^{x-1}p$

Probability density function ( PDF )

A continuous RV $X$ is described by a probability density function $f_X(x)$.
$X$ → RV
$x$ → Real #

• $\int^{\infin}_{-\infin}f_X(x)dx$ = 1
• $\int^b_af_X(x)dx$ = P[a ≤ X ≤ b]
• $P(X \in B)$ = $\int_B f_X(x)dx$
• P($x$ ≤ $X$ ≤ $\delta$) = $\int_x^{x+\delta}f_X(x)dx$
  → $\delta$ is too small
  $\approx P(X=x)$ $\approx f_X(x)*\delta$ = 0

Probability distribution function (Cumulative Distribution Function, CDF)

Def: $F_X(x) = P[X≤x] = \int^x_{-\infin}f_X(\zeta)d\zeta$
$\zeta$ → dummy variable

$f_X(x) = \frac{d}{dx}F_X(x)$

Ex_ coin tossing
$X(h) = 1, X(t) = 0$
→ $P_X(1) = P_X(0) = 1/2$

Ex_ A bus arrives at random between (0, T)
X ~ time of arrival of the bus

Properties of probability distribution function (CDF)

1. $F_X(\infin) = P[X \le \infin]$ = 1
   $F_X(-\infin) = P[X \le -\infin]$ = 0
2. $F_X(x)$ is a non-decreasing(increasing) function of $x$
   In other words, if $x_1 < x_2$, then $F_X(x_1) \le F_X(x_2)$
3. $P[X>x]$ = $1-F_X(x)$
4. $F_X(x)$ is right-continuous
5. $P[x_1<X\le x_2]$ = $F_X(x_2)-F_X(x_1)$,     $x_2>x_1$
   proof.
   { $X \le x_2$ } = { $x_1 < X \le x_2$ } $\cup$ { $X \le x_1$ }
   $P[X \le x_2]$ = $P[x_1 < X \le x_2]$ + $P[X \le x_1]$
   $F_X(x_2)$ = $P[x_1 < X \le x_2]$ + $F_X(x_1)$
   $P[x_1 < X \le x_2] = F_X(x_2)-F_X(x_1)$
6. $P[X=x] = P[x^-<X\le x]$
                        $= F_X(x)-F_X(x^-)$

Note_
1. If $F_X(x)$ is continuous at $x=x_0$, then $P[X=x_0]=0$2. If $F_X(x)$ is descontinuous at $x=x_0$, then $P[X=x_0]=F_X(x_0)-F_X(x_0^-)$

Comment_
Probability distribution function provides a " complete Statistical description " of a RV.

Typical density functions

Uniform

Gaussian ( normal )

• $X$ ~ $N$($m$ , $\sigma^2$)

• $G(x)$ = $\int^x_{-\infin}\frac{1}{\sqrt {2\pi}}e^{\zeta^2/2}d\zeta$

• $Q(x) = 1-G(x)$
  $Q(-x) = 1-Q(x)=G(x)$
  Ex_ $f_X(x)=\frac{1}{\sqrt {8\pi}}e^{-(x+3)^2/8}$
  $X$ ~ (-3, 4)

• $P[|X+3|<2] = P[-5<X<-1]$
  = $F_X(-1)-F_X(-5)$
  = $G(\frac{-1+3}{2})-G(\frac{-5+3}{2})$
  = $G(1)-G(-1)$
  = $G(1)-1+G(1)$
  = $2G(1)-1$
  = $1-2Q(1)$

Exponential

$f_X(x)=\lambda e^{-\lambda x},x\ge0$
             $=0$, otherwise

$F_X(x) = \int^x_0\lambda e^{-\lambda \zeta}d\zeta$
              $= [-e^{-\lambda \zeta}]^x_0$
              $= 1-e^{-\lambda x}, x\ge0$

Rayleigh

$f_X(x) = \frac{x}{\sigma^2}e^{-x^2/2\sigma^2}, x\ge0$

Poisson

단위 시간 안에 어떤 사건이 몇 번 발생할 것인지를 표현

• Discrete RV
• Consider a random point experiment
• $P_X(k) = (^k_n)p^kq^{n-k}$,         $p=\frac{\tau}{T}$,         $q=1-\frac{\tau}{T}$
• Assume T → $\infin$, n → $\infin$, while n/T → $\lambda$

$P_X(k) \cong e^{-\lambda\tau}\frac{(\lambda\tau)^k}{k!}$

• Q: What is the density of distance between adjacent points?$F_Y(y)=P[Y\le y]$
  $P[Y>y]=P[$ no point in interval w / length y $]$
  $1-F_Y(y)=e^{-\lambda y},y\ge0$
  $\therefore F_Y(y) = 1-e^{-\lambda y}, y\ge0$
      $f_Y(y) = \lambda e^{-\lambda y}, y\ge0$
  

---

Expectation

Definition

• For a discrete RV, $E[X] = \sum_x xP_X(x)$
  • Indicates the center of gravity of PMF
  • e.g. uniform RV

• For a continuous. RV, $E[X]=\int^\infin_{-\infin}xf_X(x)dx$
  • e.g. exponential RV

Expectation of a function of a RV

• Let $Y=g(X)$, then then $Y$ is also RV.
• For a discrete RV, $E[Y]=\sum_yyP_Y(y) = \sum_xg(x)P_X(x)$
• Similary, for a continuous RV, $E[Y]=\int^\infin_{-\infin}g(x)f_X(x)dx$

Properties

• $E[-]$ a is linear operator
• $E[aX]=aE[X]$
• $E[aX+b]=aE[X]+b$
• In general, $E[g(X)]\neq g(E[X])$
  • e.g. $g(X) = x^2$, $E[X^2]\neq E^2[X]$

Variance

• $Var[X] = E[(X-m_x)^2]$
                   $= \int^\infin_{-\infin}(x-m_x)^2f_X(x)dx$
• $\sigma^2_X = E[X^2]-m_x^2$
• $\sigma_X = \sqrt {Var[X]} ~$ ~ Standard deviation

Properties

• Variance measures the deviation of X from its mean
• $Var[-]$ is NOT a linear operator
• $Var[aX+b] = Var[Y]$
                              $= E[(Y-m_y)^2]$
                              $= E[(aX+b-am_x-b)^2]$
                              $= E[a^2(X-m_x)^2]$
                              $=a^2*Var[X]$

---

Moments

The $n^{th}$ moment of a RV $X$ : $m_n=E[X^n]=\int^\infin_{-\infin}x^nf_X(x)dx$
     → e.g. $m_1=m_x$
The $n^{th}$ central moment of a RV $X$ : $\mu_n=E[(X-m_X)^n]=\int^\infin_{-\infin}(x-m_x)^nf_X(x)dx$
     → e.g. $\mu_2=\sigma_x^2$

Ex_

---

Conditional PMF

$P_{X|A}(x|A)=P[X=x|A]$

$E[X|A]=\sum_xx*P_{X|A}(x|A)$

Conditional PDF & CDF

Conditional distribution $F_{X|A}(x|A)=P[X\le x|A]$
Conditional density $f_{X|A}(x|A)=\frac{d}{dx}F_{X|A}(x|A)$
$E[X|A]=\int^\infin_{-\infin}x*f_{X|A}(x|A)dx$

---

Total expectation theorem

• Recall that $P[B] = \sum_{i=1}^nP[B|A_i]P[A_i]$

  $E[X]= \sum_{i=1}^n E_{X|A_i}(x|A_i)*P[A_i]$

• For a discrete RV X,
  • $P_X(x) = P[X=x]=\sum_{i=1}^nP_{X|A_i}(x|A_i)P[A_i]$
  • $E[X]=\sum_xxP_X(x)$
                $= \sum_{i=1}^n \sum_xxP_{X|A_i}(x|A_i)P[A_i]$
                $= \sum_{i=1}^nE_{X|A_i}(x|A_i)*P[A_i]$
• Similary, for a continuous RV X,
  • $f_X(x)=\sum_{i=1}^nf_{X|A_i}(x|A_i)P[A_i]$
  • $E[X]=\int^\infin_{-\infin}xf_X(x)dx$
    $= \sum_{i=1}^n \int^\infin_{-\infin}xf_{X|A_i}(x|A_i)P[A_i]dx$
    $= \sum_{i=1}^n E_{X|A_i}(x|A_i)*P[A_i]$
    ~ Total expectation
    ~ Expacted value of $E_{X|A_i}(x|A_i)$

---

Memorylessness

Geometric RV

• X ~ # of independent coin tosses until first head
• $P_X(x) = (1-p)^{x-1}*p$
• $A=\{X>2\}$
• $P_{X|A}(x|A)=$ ?
• Let $Y=X-2 (Y>0)$, then $P_Y(y)=P_X(y)$
  • e.g. $P[X=5|X>2]=P[Y=3|X>2]=P[X=3]$
  • Given that $X > 2$, random variable $Y = X − 2$ has the same geometric PMF as $X$.
  • Hence, the geometric random variable is said to be memoryless, because the past has no bearing on its future behavior.

Exponential RV

• X ~ exponential RV
• $f_X(x)=\lambda e^{-\lambda x}, x>0$
• $A=\{X>2\}$
• $f_{X|A}(x|A)=\lambda e^{-\lambda x}/e^{-2\lambda}=\lambda e^{-\lambda(x-2)},x>2$
• Let $Y=X-2 (Y>0)$, then $f_Y(y)=f_X(y), y>0$
  • e.g. $P[X\le5|X>2]=P[Y\le3|X>2]=P[X\le3]$
  • Given that $X > 2$, random variable $Y = X − 2$ has the same exponential PDF as $X$.
  • Hence, the exponential random variable is also memoryless.

Ex_

---

Total probability / Bayes' theorem (continuous ver.)

$P[A] = \int^\infin_{-\infin}P[A|X=x]f_X(x)$ ~ Total prob. theorem
$f_X(x)=\frac{P[A|X=x]f_X(x)}{P[A]}=\frac{P[A|X=x]f_X(x)}{\int^\infin_{-\infin}P[A|X=x]f_X(x)}$ ~ Bayes' theorem

Coin tossing example

$P(head)=P$, with $f_P(p)$, $p\in[0,1]$
Find P(head).

$P(head|P=x)=x$
$P(head)=\int^1_0P(head|P=x)f_P(x)dx$
If $P$ is uniform on $[0,1]$
$P(head)=\int^1_0x*1dx=1/2$

HGU 전산전자공학부 이준용 교수님의 23-2 확률변수론 수업을 듣고 작성한 포스트이며, 첨부한 모든 사진은 교수님 수업 PPT의 사진 원본에 필기를 한 수정본입니다.