Machine Learning A Probabilistic Perspective 笔记

Random Values:

Concept Meaning Discrete Random Variables Probability mass function / pmf P() 判别分类器 Discriminative Classifier P(Y=c|x) 生成分类器 Generative Classifier P(Y=c|x)=P(Y=c|θ)P(x|Y=c,θ)∑c′P(Y=c′|θ)P(x|Y=c′,θ) Unconditionally / marginally independent P(x,y)=P(x)P(y) Conditionally independent P(x,y|z)=P(x|z)P(y|z) P(x,y|z)=g(x,z)h(y,z),∀x,y,z such that P(z)>0 Continuous Random Variables Cumulative distribution function / cdf F(x)=P(X≤x) Probability density function / pdf f(x)=dF(x)dx α Quantile of F F−1(α)=xα such that P(X≤xα)=α Median F−1(0.5) Lower Quantile F−1(0.25) Upper Quantile F−1(0.75) Tail area probability Mean / Expected Value

μ/EX={∑xxP(x)∫xxf(x)dx

Variance varX=E(X−μ)2 Standard Deviation stdX=varX−−−−−√

Common Distributions:

Discrete:

Name Formula Binomial Bin(k|n,θ)=(nk)θk(1−θ)n−k Bernoulli Ber(k|θ)=θk(1−θ)1−k Multinomial Mu(x|n,θ)=(nx1,...,xK)∏Kj=1θxjj,∑Kj=1xj=n Multinoulli / Categorical / Discrete Mu(x|1,θ)=∏Kj=1θxjj,∑Kj=1xj=1 Poisson Poi(x|λ)=e−λλxx! Empirical Pemp(A)=∑Ni=1wiδxi(A),0≤wi≤1,∑Ni=1wi=1,δx(A)={10if x∈Aif x∉A,D={x1,...,xN}

注：

(nk)=n!k!(n−k)!,(nx1,...,xK)=n!x1!x2!...xK!

Continuous:

Name Formula Gaussian / Normal N(x;μ,σ2)=12πσ2√e−(x−μ)22σ2,Φ(x;μ,σ2)=∫x−∞N(t;μ,σ2)dt T T(x;μ,σ2,n)=[1+1n(x−μσ)2]−n+12 Laplace Lap(x;μ,b)=12be−|x−μ|b Gamma Ga(x;shape=α,rate=β)={βαΓ(α)xα−1e−βx,0,x>0,elseα,β>0 Beta Beta(x;α,β)={1B(α,β)xα−1(1−x)β−10,0≤x≤1,elseα,β>0 Pareto Pareto(x;k,m)=kmkx−(k+1)I(x≥m)

注：

Φ(x;μ,σ2)=12[1+erf(z/2√)] where z=x−μ/σ and erf(x)=2π√∫x0e−t2dt