P1

Let $X_0, X_1, X_2, \dots, X_{N-1}$ be a random sample of $X$ having unknown mean $\mu$ and variance $\sigma^2$.
Show that the estimator of variance defined by

is an unbiased estimator of $\sigma^2$, where $\bar{X}$ is the sample mean as

---

Solve

---

---

P2

Let $X_0, X_1, X_2, \ldots, X_{N-1}$ be a random sample of a Poisson random variable $X$ with unknown parameter $\lambda$.

1. Show that the estimators $\hat{\lambda}_1$ and $\hat{\lambda}_2$ of the parameter $\lambda$ defined by

are both unbiased estimators of $\lambda$.

2. Which estimator is more efficient?

---

Solve 1

Lambda 1 is unbiased estimator

Lambda 2 is also unbiased estimator

---

Solve 2

Variance of Lambda 1

Variance of Lambda 2

Lambda 1 is more Efficient Estimator

---

---

P3

1. The data $\{x[0], x[1], \ldots, x[N-1]\}$ are observed where the $x[n]$'s are independent and identically distributed (IID) as $\mathcal{N}(0, \sigma^2)$. We wish to estimate the variance $\sigma^2$ as

2. Is this an unbiased estimator? Find the variance of $\hat{\sigma}^2$ and examine what happens as $N \to \infty$.

---

Solve 1

Unbiased Estimator

---

Solve 2

• When $N \rightarrow \infty$, $\text{Var}(\hat\sigma^2)$ goes 0

---

---

P4

Prove that the PDF of $\hat{A}$ given in Problem 3 is $\mathcal{N}(A,\sigma^2 / N)$.

---

Solve

• $E[X_i]=A$
• $\text{Var}(X_i) = \sigma^2$
• Check $E[\hat A]$
  $\hat A=\frac{1}{N}\sum_{i=1}^NX_i\\[0.3cm] E[\hat A]=E\left[\frac{1}{N}\sum_{i=1}^N X_i\right]=\frac{1}{N}\sum_{i=1}^NE[X_i]=\frac{1}{N}\cdot N\cdot A=A$
• Check $\text{Var}(\hat A)$
  $\text{Var}(\hat A)=\frac{1}{N^2}\sum_{i=1}^N\text{Var}(X_i)=\frac{1}{N^2}\cdot N \cdot \sigma^2=\frac{\sigma^2}{N}$
  $\therefore \hat A \sim \mathcal{N}\left(A,\frac{\sigma^2}{N}\right)$

---

---

P5

For the problem described in Problem 3, show that as $N \to \infty$, $\hat{A} \to A$ by using the results of Problem 4.
1. To do so, prove that

for any $\epsilon > 0$. In this case, the estimator $\hat{A}$ is said to be consistent.
2. Investigate what happens if the alternative estimator

is used instead

---

Solve 1

---

Solve 2

• New $\hat A$ is biased estimator
  $\text{Var}(\hat A)=\frac{1}{(2N)^2}\sum_{n=0}^{N-1}\text{Var}(x[n])=\frac{1}{4N^2}\cdot N \cdot \sigma^2=\frac{\sigma^2}{4N}\neq\frac{\sigma^2}{N}$

---

---

P6

This problem illustrates what happens to an unbiased estimator when it undergoes a nonlinear trasformation. In Problem 3, if we choose to estimate the unknown parameter $\theta=A^2$ by

can we say that the estimator is unbiased? What happens as $N\rightarrow \infty$?

---

Solve

• $\hat A =A$
• $\text{Var}(\hat A)=\frac{\sigma^2}{N}$
• Check $E[\hat \theta]$
  $x[n]\sim\mathcal{N}(A, \sigma^2)\\[0.2cm] E[\hat\theta]=E\left[\left(\frac{1}{N}\sum_{n=0}^{N-1}x[n]\right)^2\right]\\[0.3cm] E[\hat A^2]=\text{Var}(\hat A)+(E[\hat A])^2\\[0.2cm] E[\hat \theta]=\frac{\sigma^2}{N}+A^2\\[0.3cm] \therefore \hat \theta \text{ is biased estimator}$
• Check $E[\hat \theta]$ when $N\rightarrow \infty$
  $E[\hat \theta]=\frac{\sigma^2}{N}+A^2\rightarrow A^2\\[0.3cm] \therefore \hat \theta \text{ is Asymptatically Unbiased}$

---

---

Bonus

Unbiased Estimator for DC Level in White Gaussian Noise

Consider the observation

where $A$ is the parameter to be estimated and $w[n]$ is WGN

• The parameter $A$ can take on any value in the interval $-\infty<A<\infty$
• Then, a reasonable estimator for the average value of $x[n]$ is
  $\hat A=\frac{1}{N}\sum_{n=0}^{N-1}x[n]$
  or the sample mean.
• Due to the linearity properties of the expectation operator
  $E[\hat A]=E\left[\frac{1}{N}\sum_{n=0}^{N-1}x[n]\right]\\[0.3cm] =\frac{1}{N}\sum_{n=0}^{N-1}E[x[n]]\\[0.3cm] =\frac{1}{N}\sum_{n=0}^{N-1} A\\[0.3cm] =A$
  for all $A$
  $\therefore \text{The sample mean is unbiased}$

All Content has been written based on lecture of Prof. eui-seok.Hwang in GIST(Detection and Estimation)