Terminology for Hypothesis testing

Suppose $C$ means critical region

• decision function

  $\phi(\underline{X})=$ $\left\{\begin{matrix} 1 & \underline{X} \in C\\ 0 & \underline{X} \in C^c \end{matrix}\right. \\ where \space C$: critical region

• Type 1 error probability

  $P_\theta(\phi(\underline{X})=1)=P_\theta(\underline{X} \in C)$ for $\theta \in \Omega_0$
  ($\Omega_0$ means parameter space in null hypothesis)
  => null hypothesis 가 참인데 null hypothesis 기각할 확률

• Type 2 error probability

  $P_\theta(\phi(\underline{X})=0)=P_\theta(\underline{X} \notin C)$ for $\theta \in \Omega_1$
  ($\Omega_1$ means parameter space in alternative hypothesis)
  => alternative hypothesis 가 참인데 null hypothesis 기각하지 못할 확률

• power function

  $\beta(\theta)=P_\theta(\underline{X} \in C)$
  If $\theta \in \Omega_0,$ $\beta(\theta)=$ Type 1 error prob.
  If $\theta \in \Omega_1,$ $\beta(\theta)=1-$Type 2 error prob. (a.k.a power)

ex1) $X_1,...,X_{100} \overset{iid}{\sim} N(\mu,1),$ $H_0:\mu=0$ vs $H_1:\mu=1$
Let $\phi(\underline{X})=\left\{\begin{matrix} 1 & \overline{X}>0.196\\ 0 & \overline{X} \leq 0.196 \end{matrix}\right.$
$\overline{X} \sim N(\mu,\frac{1}{100})$

Type 1 error prob. :
$P_{\mu=0}(\overline{X}>0.196)=P_{\mu=0}(\frac{\overline{X}}{1/ \sqrt{100}} > \frac{0.196}{1/ \sqrt{100}})=P(Z>1.96)$

Type 2 error prob. :
$P_{\mu=1}(\overline{X} \leq 0.196)=P_{\mu=1}(\frac{\overline{X}-1}{1/\sqrt{100}} \leq \frac{0.196-1}{1\sqrt{100}})=P(Z \leq -8.04)$

$\beta(\mu)=P_\mu(\overline{X}>0.196)=P_\mu(10(\overline{X}-\mu) > 10(0.196-\mu))=P(Z>10(0.196-\mu))=1-\Phi(10(0.196-\mu))$

• size $\alpha$ test

  $\phi(\theta)$ is size $\alpha$ test if $\alpha=sup_{\theta \in \Omega_0} \beta(\theta)$
  $\phi(\theta)$ is level $\alpha$ test if $\alpha \geq sup_{\theta \in \Omega_0} \beta(\theta)$

• Most Powerful test

  Suppose $X_1,...,X_n \overset{iid}{\sim} f(x;\theta),\theta \in \Omega$
  $\phi$ is called a most powerful test of size $\alpha$ for testing $H_0:\theta=\theta_0$ vs $H_1:\theta=\theta_1$ if
  ① $\beta_\phi(\theta_0)=\alpha$
  ② $\beta_\phi(\theta_1) \geq \beta_{\phi'}(\theta_1)$ for any $\phi'$ satisfying $\beta_{\phi'}(\theta_0)=\alpha$
  
  Type 2 error 를 최대한 줄인 테스트를 찾는 거임