Kalman Filter

1.1 Linear Optimal Filtering

LOF是kalman filter的精髓，可以描述为：

• inputs: x[0], x[1], …, x[n]
• outputs: y[0], y[1], …, y[n]
• filter: w[0], w[1], …, w[n]
• objection function: en=yn−ŷ n, ŷ nspanned by{xn}

1.2 Orthogonality Principle

LOF得到的ŷ n垂直于yn−ŷ n

LOF ↔ŷ n⊥[yn−ŷ n]↔E[ŷ n[yn−ŷ n]T]=0

1.4 State Model

Xk+1YK=FkXk+GKUK=HkXk+Bkenvolution of statesobservations(1)(2)

• Hk,Fk,Gkare determinated and known, MATRIX
• Uk,Bk: White Noises of state and observation

这里B加粗B表示B是向量。

1.5 Objective And Hypothesis

Objective:

Obtain recursively linear estimators of Xkbased on the observations up to the k instants ({Yn}n=0,...,k) and the signal statistical properties.

Hypothesis:

E[X0]=x¯0,E[Bk]=0,E[Uk]=0E⎡⎣⎢⎢⎡⎣⎢⎢BkX0Uk⎤⎦⎥⎥[BTlXT0UTl]⎤⎦⎥⎥RkQkδkl=⎡⎣⎢⎢Rkδkl000P0000Qkδkl⎤⎦⎥⎥=E[BkBTk]=E[UkUTk]={1,0,if k=l;otherwise.

这里要证明一下，为什么假设E[Bk]=0,E[Uk]=0会是『Without loss of generality 』:
假设Bk=B′k+B0, Uk=U′k+U0

Yk(Yk−B0)Yknew(Xk+1+C)C=HkXk+B′k+B0=HkXk=(Yk−B0)=HkXk=Fk(Xk+C)+GKU′k=(Fk−1)−1GkU0

Remark

Stationary: 是指随机过程（不是随机变量）的联合概率密度不随时间改变而改变的状态。

Stationary Process: is a stochastic process whose joint probability distribution does not change when shifted in time. Consequently, parameters such as mean and variance, if they are present, also do not change over time. – From Wiki

须满足

E[Xk]E[XkXTl]=x¯=Γk−l

Proof: Xk,Ykare non-stationary unless

1. the matrices Hk,Fk,Gk,Rk,Qk are time invariant (noted by H, F, G, R and Q);
2. F is a stable “filter”, i.e., all eigenvalues lies within the unit circle;
3. x¯0 = 0 and the initial covariance P0is a solution to the Lyapunov equation:
   P=FPFT+GQGT

1.3 Notations

• yn: space spanned by {Yi}i=1,..,n;
• X̂ (n+1|yn): optimal linear estimate of Xn+1 knowing yn;
• Ŷ (n|yn−1): optimal linear predictor of Yn knowing yn−1;
• α[n] =Yn−Ŷ (n|yn−1): innovation/error, the information we have gained.

Remark

1. i = n, X̂ (n|yn−1) is the Kalman Filter (On line)
2. i < n, X̂ (i|yn−1) is the Kalman Smoothing (Off line)

α[n]X̂ (n+1|yn)Ŷ (n|yn−1)=Yn−Ŷ (n|yn−1)∈{y1:n}∈{y1:n−1}

Proposition

1. α[n]⊥yn−1↔E[α[n]YTk]=0, 1≤k<n
2. α[n]⊥α[k]↔E[α[k]α[k]T]=0, k≠n
3. Linear transformation
   yn={Y1,...,Yn}↔{α[1],...,α[1]}

证明：
1.

sinceand⇒:Ŷ (n|yn−1)is optimal linear:α[n]=Yn−Ŷ (n|yn−1):α[n]⊥yn−1

这里有一些引理：

spaceifif:A,B,C,D,A⊥B:C⊆B⇒A⊥C:B=C∪D⇒A⊥C&A⊥D

sinceand⇒:Ŷ (n|yn−1)is optimal linear:α[n]=Yn−Ŷ (n|yn−1):α[n]⊥yn−1

sincesincesincesince:Yn−1=Hn−1Xn−1+Bn−1:Xn=FnXn−1+Gn−1Un−1⇒{Xn}space⊂{X0,U0:n−1}space⇒{Yn}space⊂{X0,U0:n−1,B0:n}space:α[n]⊥yn−1⇒α[n]⊥{X0,U0:n−2,B0:n−1}space:{X0,U0:n−3,B0:n−2}space⊂{X0,U0:n−2,B0:n−1}space⇒α[n]⊥{X0,U0:n−3,B0:n−2}⇒α[n]⊥yn−2⇒α[n]⊥yi,0 < i < n

◻
2.

sincesince⇒:α[k]=Yk−Ŷ (n|yk−1)∈{Yk}space:α[k]⊥yk:α[n]⊥α[k],1≤k≤n−1

3.

α[n]⇒α[n]Yn⇐Yn−1=Yn−Ŷ (n|yn−1)∈{Yn}=α[n]+Ŷ (n|yn−1)∈{α[n],Yn−1}∈{α[n−1],Yn−2}归纳法

Conclusion of Space Relationship

{Xn}space{Yn}spaceBkBkUkUkBk⊂{X0,U0:n−1}space⊂{X0,U0:n−1,B0:n}space⊥Ul⊥Xl,0≤l≤k⊥Xl,0≤l≤k⊥Yl,0≤l≤k⊥Yl,0≤l≤k−1

The Innovation Covariance

ϵ(n,n−1)K(n,n−1)Ŷ (n|yn−1)α[n]=Xn−X̂ (n|yn−1)=E[ϵ(n,n−1)ϵ(n,n−1)T]=HnX̂ (n|yn−1)=Yn−Ŷ (n|yn−1)the prediction errorcovariancethe prediction of observationinnovation

Proof the prediction of observation:

Ŷ (n|yn−1)=HnX̂ (n|yn−1)

Proof: 既然Ŷ (n|yn−1)表示 optimal linear predictor of Yn knowing yn−1, 那么它与 Y的差必定与空间spanned by{yn−1}相互垂直.

Yn−Ŷ (n|yn−1)  ⇒E[(Yn−Ŷ (n|yn−1))(yn−1)T]since  HnXn+Bn⇒  E[(Hn(Xn−X̂ (n|yn−1))+Bn)yTn−1]⇐Xn−X̂ (n|yn−1)⊥yn−1(Optimal ⊥  yn−1=0=Yn=0Filter properties) and Bn⊥yn−1

◻