Population $X$ & Individual
Finite population & Infinite individual
Simple random sample
$X_1, X_2, …, X_n$ are independent and identically distributed with $X$, then call ****$X_1, X_2, …, X_n$ (simple random) samples from $X$, $n$ sample size, call sample observations $x_1, x_2, …, x_n$ sample values.
Statistic
****$X_1, X_2, …, X_n$ are samples from $X$, $g(X_1, X_2, …, X_n)$ are a function of ****$X_1, X_2, …, X_n$ with no unknown parameters, then $g(X_1, X_2, …, X_n)$ is a statistic, the distribution of a statistic is called sampling distribution.
Common Statistics
(1) sample mean: $\bar X =\frac{1}{n}\sum^n_{i=1}X_i$
(2) sample variance: $S^2 = \frac{1}{n-1} \sum^n_{i=1} (X_i-\bar X)^2 =\frac{1}{n-1} (\sum^n_{i=1} X_i^2 - n \bar X ^2)$
standard deviation: $S = \sqrt{S^2}$
Why is $S^2$ defined so? The expectation of the sample variance must be the same as the population variance. $S^2 = \frac{1}{n-1} (\sum^n_{i=1} X_i^2 - n \bar X ^2)$ then $E((n-1)S^2) = \sum^n_{i=1}E(X_i^2) - nE(\bar X^2)= \sum^n_{i=1}(DX_i+E^2X_i)-n(D\bar X+E^2\bar X)$ $=\sum^n_{i=1}(\sigma^2+\mu^2)-n(D(\frac{1}{n}\sum^n_{i=1}X_i)+E^2(\frac{1}{n}\sum^n_{i=1}X_i)) =\sum^n_{i=1}(\sigma^2+\mu^2)-n(\frac{\sigma^2}{n}+\mu^2)=(n-1)\sigma^2$ therefore $E(S^2) = \sigma^2$.
From above, we have a conclusion in below: $E\bar X = EX =\mu, D\bar X = \frac{1}{n}DX = \frac{\sigma^2}{n}, E(S^2) =DX= \sigma^2$.
(3) $k$-order origin moment of sample: $A_k=\frac{1}{n}\sum^n_{i=1} X_i^k$
$\bar X = A_1$
(4) $k$-order central moment of sample: $B_k = \frac{1}{n}\sum^n_{i=1} (X_i-\bar X)^k$
$S^2 = \frac{n}{n-1} B_2>B_2$
$\chi^2 \sim \chi^2(n)$
$X_1,…,X_n$ are independent and obey $N(0,1)$, then $\chi^2 = X_1^2 +…+X_n^2$ obey $\chi^2$ distribution with degree of freedom $n$.
specially, if $X\sim N(0,1)$, then $X^2 \sim \chi^2(1)$.
Upper $\alpha$ quantile of $\chi^2$ distribution
$\chi^2_\alpha(n)$: $P\{ \chi^2>\chi^2_\alpha(n)\} = \alpha$
$f(x)$ of $\chi^2$ distribution is unimodal and asymmetric.