若 A1,A2,A3 相互独立，则 A1 与 A2∪A3 相互独立。
证明
P(A1(A2∪A3))=P(A1A2∪A1A3)
=P(A1A2)+P(A1A3)−P(A1A2A3)
=P(A1)P(A2)+P(A1)P(A3)−P(A1)(A2A3)
=P(A1)[P(A2)+P(A3)−P(A2A3)]
=P(A1)P(A2∪A3)
Discrete Distributions
Binomial Distribution
Bin(k;n,θ)=(nk)θk(1−θ)n−k,k=0,...,n
Bernoulli Distribution
Ber(x;θ)=θk(1−θ)1−k,k=0,1
Multinomial Distribution
Mu(x|n,θ)=(nx1,⋯,xK)∏Kj=1θxjj,x=⎛⎝⎜⎜x1⋮xK⎞⎠⎟⎟
Multinoulli Distribution
Mu(x|1,θ)=∏Kj=1θxjj
Poisson Distribution
Poi(k|λ)=e−λλkk!
Empirical Distribution / Measure
Pemp(A)=1|D|∑x∈DΠ(x∈A)
Continuous Distributions
Gaussian Distribution
N(x;μ,σ2)=12πσ2−−−−√e−12σ2(x−μ)2
Φ(x;μ,σ2)=∫x−∞N(t;μ,σ2)dt
Φ(x;μ,σ2)=12[1+erf(z/2√)],where
z=x−μσ
erf(x)=2π√∫x0e−t2dt
Student t Distribution
T(t;μ,σ2,n)=1nπ−−−√Γ((n+1)/2)Γ(n/2)[1+1n(x−μσ)2]−n+12
F Distribution
U∼χ2(n1),V∼χ2(n2)
F=U/n1V/n2∼F(n1,n2):
f(x;n1,n2)=1B(x;n12,n22)(n1n2)n12xn12−1(1+n1n2x)n1+n22Π(x≥0),where
B(a,b)=Γ(a)Γ(b)Γ(a+b)
Laplace Distribution
Lap(x;μ,b)=12bexp(−|x−μ|b)
Gamma Distribution
Ga(x;shape=α,rate=β)=βαΓ(α)xα−1e−βxΠ(x≥0)
Γ(x)=∫+∞0tx−1e−tdt
Exponential Distribution
Expon(x;λ)=Ga(x;1,λ)=λe−λxΠ(x≥0)
Erlang Distribution
Erlang(x;λ)=Ga(x;2,λ)
Chi-squared Distribution
χ2(x;n)=Ga(x;n2,12)=2−n2Γ(n2)xn2−1e−12xΠ(x≥0)
Inverse Gamma Distribution
If X∼Ga(x;shape=α,rate=β),then1X∼IG(x;shape=α,scale=β):
IG(x;shape=α,scale=β)=βαΓ(α)x−(α+1)e−β/xΠ(x>0)
Beta Distribution
Beta(x;a,b)=1B(a,b)xa−1(1−x)b−1,where
B(a,b)=Γ(a)Γ(b)Γ(a+b)
Pareto Distribution
Pareto(x;k,m)=kmkx−(k+1)Π(x≥m)
Multivariate Gaussian
N(x;μ,Σ)=1(2π)D/2|Σ|1/2exp[−12(x−μ)⊺Σ−1(x−μ)]
Multivariate Student t Distribution
T((x;μ,Σ,n)=|Σ|−1/2(nπ)D/2Γ((n+D)/2)Γ(n/2)[1+1n(x−μ)⊺Σ−1(x−μ)]−n+D2
Σ:Scalematrix
Dirichlet Distribution
Dir(x;α)=1B(α)∏Kk=1xαk−1kΠ(x∈SK)
SK={x:0≤xk≤1,∑Kk=1xk=1}
Multivariate change of variables
fy(y)=fx(x)|det(∂x∂y)|
