Sufficiency

Def)
A statistic $T(\underline{X})$ is a sufficient statistic for $\theta$ if the conditional distribution of $\underline{X}$ given $T(\underline{X})$ does not depend on $\theta.$

ex1) $X_1,X_2 \overset{iid}{\sim} b(1,\theta)$
① $T_1(X_1,X_2) = X_1 \\ => P(X_1,X_2 | T_1)=\frac{P(X_1=x_1, X_2=x_2, X_1=t_1)}{P(X_1=t_1)} = \frac{P(X_1=t_1, X_2=x_2)}{P(X_1=t_1)} = P(X_2=x_2) = \theta^{x_2} (1-\theta)^{1-x_2}$: depends on $\theta$

② $T_2(X_1,X_2)=X_1 + X_2 \\ => \frac{P(X_1=x_1, X_2=x_2, X_1+X_2=t_2)}{P(X_1+X_2=t_2)} = \frac{P(X_1=x_1)P(X_2=t_2-x_1)}{P(X_1+X_2=t_2)}=\frac{\theta^{x_1} (1-\theta)^{1-x_1} \theta^{t_2-x_1} (1-\theta)^{1-t_2+x_1}}{ \begin{pmatrix} 2 \\ t_2 \end{pmatrix} \theta^{t_2} (1-\theta)^{2-t_2}} = \frac{1}{\begin{pmatrix} 2 \\ t_2 \end{pmatrix}}$: does not depend on $\theta$

Theorem)
$T(\underline{X})$ is a sufficient statistic if $\frac{P(x_1,...,x_n;\theta)}{g(t;\theta)}$ does not depend on $\theta$

$P(x_1,...,x_n;\theta)$: joint pdf or pmf of $\underline{X}$
$g(t;\theta)$: pdf or pmf of $T(\underline{X})$

(If $g$: 1-1 function, $T$: sufficient statistic => $g(T)$: sufficient statistic)

pf)

ex1) $X_1,...,X_n \overset{iid}{\sim} f(x)$
order statistic $X_{(1)},...,X_{(n)}$ are sufficient statistic for $f$
$\frac{f_{\underline{X}}(\underline{x})}{f_{\underline{Y}}(\underline{y})}=\frac{f(x_1)...f(x_n)}{ \frac{n!}{1...1} f(y_1)...f(y_n)}=\frac{1}{n!}(b/c \space \prod_{i=1}^{n} f(x_i)=\prod_{i=1}^{n} f(y_i))$

Theorem) Factorization theorem
$\forall x,\theta$,
$T(\underline{X})$: sufficient statistic for $\theta$ <=> $f(\underline{x};\theta)=g(t;\theta)h(\underline{x})$
pf)

ex1) $X_1,...,X_n \overset{iid}{\sim} N(\mu,1)$
$f(\underline{x};\mu) = \prod_{i=1}^{n} \frac{1}{\sqrt{2 \pi}} exp(-\frac{(x_i-\mu)^2}{2}) = (\frac{1}{\sqrt{2 \pi}})^n exp(-\frac{1}{2} \sum_{i=1}^{n} (x_i-\mu)^2) = (\frac{1}{\sqrt{2 \pi}})^n exp(-\frac{1}{2} \sum_{i=1}^{n} x_i^2) exp(-\frac{1}{2}(n \mu^2 - 2 \mu \sum_{i=1}^{n} x_i)) \\ \\ h(\underline{x})= (\frac{1}{\sqrt{2 \pi}})^n exp(-\frac{1}{2} \sum_{i=1}^{n} x_i^2) \\ g(t;\mu) = exp(-\frac{1}{2}(n \mu^2 - 2 \mu \sum_{i=1}^{n} x_i)) \\ => T=\sum_{i=1}^{n} x_i$ : sufficient statistic for $\mu$

ex2) $X_1,...,X_n \overset{iid}{\sim} U[0,\theta]$
$f(\underline{x};\theta)=\prod_{i=1}^{n} \frac{1}{\theta} I_{(0 \leq x_i \leq \theta)} = (\frac{1}{\theta})^n \prod_{i=1}^{n} I_{(x_i \leq \theta)} \prod_{i=1}^{n} I_{(0 \leq x_i)} \\ \\ h(\underline{x}) = \prod_{i=1}^{n} I_{(0 \leq x_i)} \\ g(t;\theta) = (\frac{1}{\theta})^n \prod_{i=1}^{n} I_{(x_i \leq \theta)} \\ => T=\prod_{i=1}^{n} I_{(x_i \leq \theta)}$ : sufficient statistic for $\theta$

Theorem)
If $f(x;\underline{\theta})$ belongs to exponential family s.t. $f(x;\underline{\theta}) = g(x) c(\underline{\theta}) exp(\sum_{j=1}^{k} w_j(\underline{\theta}) t_j(x)) \space (j \leq k)$,
$(\sum_{i=1}^{n} t_1(x_i),...,\sum_{i=1}^{n} t_k(x_i))$: sufficient statistic for $\underline{\theta}$
pf)