1.Unbiased estimation

$\hat \theta$ is an unbiased estimation of $\theta$, if $E(\hat \theta) = \theta ,\forall \theta \in \Theta.$

if $E(\hat \theta) ≠ \theta$, call $E(\hat \theta) - \theta$ the bias of $\hat \theta$.

if $E(\hat \theta) ≠ \theta$ but $\lim_{n→ +\infty}E(\hat \theta) = \theta$, then call $\hat \theta$ an asymptotic unbiased estimation of $\theta$.

<aside> ✔️ e.g.5 $X\sim f(x)= \frac{1}{2\mu}e^{-\frac{|x|}{\mu}}, -\infty<x<+\infty$, $X_1, X_2, …, X_n$ are samples, solve the MLE of $\mu$, and prove that it’s an unbiased estimation of $\mu$. $L(\mu) = \prod^n_{i=1} \frac{1}{2\mu}e^{-\frac{|x|}{\mu}}=2^{-n}\mu^{-n}e^{-\frac{1}{\mu}\sum^n_{i=1}|x_i|}, l(\mu) = -n\ln2-n\ln\mu-\frac{1}{\mu}\sum^n_{i=1}|x_i|.$ $\frac{dl(\mu)}{d\mu}= -\frac{n}{\mu}+\frac{1}{\mu^2}\sum^n_{i=1}|x_i|=0\Rightarrow \hat{\mu}L= \frac{1}{n}\sum^n{i=1}|X_i|.$ $E(\hat{\mu}L)=E(\frac{1}{n}\sum^n{i=1}|X_i|)= \frac{1}{n}\sum^n_{i=1}E|X_i|=E|X| = \int^{+\infty}{-\infty}|x|f(x)dx = \int^{+\infty}{-\infty}|x|\frac{1}{2\mu}e^{-\frac{|x|}{\mu}}dx=\mu.$ Therefore it’s unbiased.

</aside>

<aside> <img src="/icons/light-bulb_lightgray.svg" alt="/icons/light-bulb_lightgray.svg" width="40px" /> e.g.6 Test the unbiasedness of moment estimator $\hat \theta = 2\bar X$ and MLE $\hat \theta_L = max\{X_1, X_2, …, X_n\}$ of parameter $\theta$ of population $X\sim U[0,\theta]$. $E(\hat \theta) = E( 2\bar X) = 2E(\bar X)=2E(X) = 2\cdot \frac{\theta}{2}=\theta,$ therefore it’s unbiased. $E(\hat \theta_L)=E(max\{X_1,…,X_n\})$, let $Y=max\{X_1, …,X_n\}$, $X\sim U[0,\theta]$, therefore $F_Y(y)= P\{Y\le y\}= [F_X(y)]^n= \left \{ \begin{array} {lc} 0 &, y<0 \\ \frac{y^n}{\theta^n} & ,0\le y\le\theta \\ 1 & y>\theta \end{array} \right.$ $f_Y(y) =F'_Y(y) = \left \{ \begin{array} {lc} \frac{ny^{n-1}}{\theta^n} & ,0\le y\le \theta \\ 0 & , \text{elsewhile} \end{array} \right., E(\hat \theta_L)=\int^\theta_0y\frac{ny^{n-1}}{\theta^n}dy = \frac{n}{n+1}\theta$, therefore it’s not unbiased.

</aside>

Correcting

1.if $E(\hat \theta) =a\theta +b , a\ne 0,$ then $\frac{\hat\theta -b}{a}$ is an unbiased estimation.

therefore, for e.g.6, $Y=\frac{n+1}{n}max\{X_1, ...X_n\}$ is an unbiased estimation.

2.if $\hat \theta$ is an unbiased estimation of $\theta$, and $D(\hat \theta)>0,$ then $(\hat\theta)^2$ is not unbiased.

Proof: $E(\hat \theta)^2 = D(\hat \theta)+E^2(\hat \theta)>E^2(\hat \theta)=\theta^2.$

2.Valid estimation

$\hat\theta_1, \hat\theta_2$ are two different unbiased estimator of $\theta$, is $\forall \theta \in\Theta, D(\hat\theta_1)\le D(\hat\theta_2)$, then we call $\hat \theta_1$ is more valid than $\hat \theta_2$.

<aside> <img src="/icons/light-bulb_lightgray.svg" alt="/icons/light-bulb_lightgray.svg" width="40px" /> e.g.7 for samples $(X_1, X_2, …,X_n)$ in population $X$, $X_i, \bar X$ are both unbiased estimator of $\mu = E(X)$, but $D(X_i) = D(X) = \sigma^2, D(\bar X)=\frac{1}{n}\sigma^2$, therefore $\bar X$ is more valid.

</aside>

<aside> <img src="/icons/light-bulb_lightgray.svg" alt="/icons/light-bulb_lightgray.svg" width="40px" /> e.g.8 $X\sim U[0, \theta]$, $(X_1, X_2, …,X_n)(n \ge 2)$ are samples, given two unbiased estimation of $\theta$: $\hat\theta_1=2\bar X, \hat\theta_2 = \frac{n+1}{n}max\{X_1,X_2,…,X_n\}$(see e.g.6), tell which one is more valid. $D(\hat \theta_1) = D(2\bar X) = \frac{4}{n}D(X) = \frac{\theta^2}{3n}$. $D(\hat\theta_2) = D(\frac{n+1}{n}max\{X_1, X_2, ...,X_n\} )=(\frac{n+1}{n})^2D(Y)=\frac{\theta^2}{n(n+2)}.$ $n≥2,$ therefore $D(\hat\theta_1)>D(\hat \theta_2), \hat\theta_2$ is more valid.

</aside>

3.Mean Square Error Principle

$\hat \theta$ is a point estimation of $\theta$, and variation exists, then call $E(\hat \theta- \theta)^2$ the mean square error of the estimator $\hat\theta$, which is $Mse(\hat\theta)=Mse(\theta,\hat\theta)=Mse_\theta(\hat\theta)$. Mean square error principle: the smaller $Mse(\hat \theta) = E(\hat \theta-\theta)^2$ is, the better estimator $\hat \theta$ is.

if $\hat \theta$ is an unbiased estimator of $\theta$, then $Mse(\hat\theta) = E(\hat \theta - E(\hat\theta))^2 =D(\hat \theta).$

<aside> <img src="/icons/light-bulb_lightgray.svg" alt="/icons/light-bulb_lightgray.svg" width="40px" /> e.g.9 Use sample variation $S^2$ and 2-order central moment $B_2$ to estimate the variation $\sigma^2$ of a normal population, by mean square error principle, tell which one is better. $X\sim N(\mu, \sigma^2)$, then $\frac{(n-1)S^2}{\sigma^2}\sim\chi^2(n-1), E(S^2)=\sigma^2$, therefore, $Mse(S^2)=E(S^2-\sigma^2)^2 = D(S^2) =\frac{\sigma^4}{(n-1)^2}D(\frac{(n-1)S^2}{\sigma^2})=\frac{2\sigma^4}{n-1}.$ $Mse(B_2)= E(B_2-\sigma^2)^2=D(B_2-\sigma^2)+E^2(B_2-\sigma^2)\overset{\sigma^2\text{is a constant}}=D(B_2)+[E(B_2)-\sigma^2]^2$ $= D(\frac{n-1}{n}S^2)+[E(\frac{n-1}{n}S^2)-\sigma^2]^2=\frac{(n-1)^2}{n^2}D(S^2)+(\frac{n-1}{n}\sigma^2-\sigma^2)^2=\frac{2n-1}{n^2}\sigma^4.$ $n>1$, therefore $Mse(B_2)<Mse(S^2)$, $B_2$ is a better estimator.

</aside>

4.Consistent estimation

$\hat \theta$ is an estimation of parameter $\theta$, if $\forall \theta \in \Theta, n \rarr +\infty, \hat \theta_n \overset{P}\rarr \theta,$ then call $\hat \theta$ a consistent estimation of $\theta$.