Mathematical expectation

$E(X) = \sum_{i}x_ip_i$ (discrete) $= \int^{+\infty}_{-\infty} xf(x)dx$ (continuous)

$→ Y=g(X),$ then $E(Y) = \sum_{i}g(x)p_i$ (discrete) $= \int^{+\infty}_{-\infty} g(x)f(x)dx$ (continuous)

For $E(X)$ to exist, $\sum_{i}x_ip_i$ must be an absolutely convergence series, otherwise $E(X)$ would change if we alter the order of addition. n.e.1. Cauchy distribution $f(x) = \frac{1}{\pi (1+x^2)}, -\infty <x<+\infty$ $E(X)= \int^{+ \infty}{-\infty}xf(x)dx = 0$, but $\int^{+ \infty}{-\infty}|x|f(x)dx = \frac{2}{\pi}\int^{+ \infty}0 \frac{x}{1+x^2}dx=\frac{1}{\pi}\ln(1+x^2)|^\infty_0=\infty$, therefore it doesn’t exist. n.e.2. $P(X=(-1)^{k+1}\frac{3^k}{k})=\frac{2}{3^k}, k=1,2,...$ $\sum p = \frac{2}{3} \frac{1-(\frac{1}{3})^n}{1-\frac{1}{3}} =1$, therefore the definition is valid. $E(X) = \sum{k=1} (-1)^{k+1}\frac{3^k}{k} \frac{2}{3^k}=2\sum_{k=1}\frac{(-1)^{k+1}}{k}=2\ln 2$, but it’s obviously not absolutely convergent.

$(X,Y)$ is a bivariate random variable with density function $f(x,y)$

$Z = g(X,Y),$ then $E(Z) = \sum_{i}\sum_{j}g(x_i, y_j)p_{ij}$ (discrete) $= \int^{+\infty}{-\infty}\int^{+\infty}{-\infty}g(x, y)f(x,y)dxdy$ (continuous)

$E(X) = \int^{+\infty}{-\infty}\int^{+\infty}{-\infty}xf(x,y)dxdy,E(Y) = \int^{+\infty}{-\infty}\int^{+\infty}{-\infty}yf(x,y)dxdy$

$X<Y$ are independent $\Longrightarrow$ $X, Y$ are irrelevant $\iff E(XY)=E(X)E(Y)$

Variance

$D(X) = Var(X) = E(X-EX)^2 = EX^2 -E^2(X)$

If $X,Y$ are independent, $D(XY) = E(XY)^2 - E^2(XY) = EX^2\cdot EY^2 - E^2X\cdot E^2Y$

Covariance

$Cov(X,Y) = E[(X-EX)(Y-EY)] = E(XY)-EX\cdot EY$

Derivation: If $X,Y$ are independent, $E(XY)=E(X)E(Y)$, which is $E[(X-EX)(Y-EY)]= 0$. therefore $Cov(X,Y)$ is defined.

(1)$Cov(X,Y) = Cov(Y,X), Cov(X, X)=D(X)$

(2)$Cov(aX+bY+c,Z) = aCov(X, Z)+bCov(Y,Z)$.

<aside> ✔️ e.g.1 Simplify $Cov(X+Y,X-Y)$.

</aside>

(3) $(Cov(X,Y))^2 \le D(X)D(Y)$, “=” holds iff $Y= c_1 +c_2X$.

Correlation coefficient

$\rho_{XY} = \frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}} = Cov(X^,Y^)=Cov(\frac{X-EX}{\sqrt{Var(X)}},\frac{Y-EY}{\sqrt{Var(Y)}})$

(1) $|\rho_{XY}|\le 1$;

(2) $\rho_{XY} =0 \iff Cov(X,Y) =0 \iff E(XY) =E(X)E(Y) \iff D(X \pm Y) = DX+DY$

(3) $\rho_{XY}=\pm1 \iff Y=\pm aX+b (a>0)$

$\rho$ with $EX, DX$