Method of Moments(MoM) estimator

$\hat{m}_1 = m_1(\theta_1,...,\theta_k) \\ \vdots$
$\hat{m}_k = m_k(\theta_1,...,\theta_k)$
where $\hat{m}_j = \frac{1}{n}\sum_i (X_i)^j$ and $m_j=E_{\theta}(X^j)(j=1,...,k)$
위 연립방정식을 풀어서 나오는 해가 MoM estimator

ex1) $X_1,...,X_n \overset{iid}{\sim} N(\mu,\sigma^2)$
$E(X)=\mu \space \space => \space \space \hat{\mu}=\overline{X}$
$E(X^2)= \sigma^2 + \mu^2 \space \space => \space \space \hat{\sigma}^2 = \overline{X^2} - \overline{X}^2$

ex2) $X_1,...,X_n \overset{iid}{\sim} Poisson(\lambda)$
$\hat{\lambda} = \overline{X} \\ E(X^2)= \lambda+\lambda^2 = \overline{X^2}$
$\hat{\lambda} = \overline{X}$ 이므로 $E(X^2)= \overline{X}^2 + \overline{X}$ 으로도 표현될 수 있다.
하지만 $\overline{X}^2 + \overline{X} \neq \overline{X^2}$
=> MoM estimator 는 unique 하지 않다는 문제점이 있다.

Maximum Likelihood Estimator(MLE)

Def)
MLE of $\theta$ is the maximizer of $L(\theta;\underline{x})$

ex1) $X_1,...,X_n \overset{iid}{\sim} N(\theta,1)$

ex2) $X_1,...,X_n \overset{iid}{\sim} N(\theta_1,\theta_2)$

ex3) $X_1,...,X_n \overset{iid}{\sim} U[\theta-0.5,\theta+0.5]$

Theorem) functional invariance of MLE
Let $\eta=g(\theta)$ be a parameter of interest.
If $\hat{\theta}$ is the MLE of $\theta$, then $\hat{\eta}=\widehat{g(\theta)}=g(\hat{\theta})$ is MLE of $\eta$

ex1) $X_1,...,X_n \overset{iid}{\sim} b(1,p)$ and $\eta=\frac{p}{1-p}$
MLE of $\eta?$
$\hat{p} = \overline{X} \space \space => \space \space \hat{\eta}=\frac{\overline{X}}{1-\overline{X}}$ by functional invariance of MLE