[LA] Centering a data set

1. Centering data set

If we have a data set X∈Rn×p(each row is a sample), then column mean of this data set can be expressed in

X¯=1nXT1n

So the centered data set is

Xc=X−1nX¯T=X−1n1n1TnX=(I−1n1n1Tn)X

The matrix

C=(I−1n1p1Tp)

is called centering matrix.

2. Application

Note that this is a more complex decomposition by centering matrix

Proof of Proposition 1 in
http://blog.csdn.net/comeyan/article/details/50514596

proof: Firstly we express mean of full data set by group means,

μ¯=∑g=1NngNμ¯g=1N(n1−−√μ¯1,n2−−√μ¯2,⋯,ng−−√μ¯G)(n1−−√,n2−−√,⋯,nG−−−√)T

Let K=(n1−−√,n2−−√,⋯,nG−−−√)T, then using the formula of between covariance matrix, we have

(n1−−√μ¯1−n1−−√μ¯,n2−−√μ¯2−n2−−√μ¯,⋯,nG−−−√μ¯G−nG−−−√μ¯)=(n1−−√μ¯1,n2−−√μ¯2,⋯,nG−−−√μ¯G)−μ¯(n1−−√,n2−−√,⋯,nG−−−√)=(n1−−√μ¯1,n2−−√μ¯2,⋯,nG−−−√μ¯G)(I−1NKKT)

ng−−√μ¯g=ng−−√1ngXT1g=1ng−−√XT1g

Σ^b=1N∑g=1Gng(μ¯g−μ¯)(μ¯g−μ¯)T=1N∑g=1Gng−−√(μ¯g−μ¯)ng−−√(μ¯g−μ¯)T=1N(n1−−√μ¯1,n2−−√μ¯2,⋯,nG−−−√μ¯G)(I−1NKKT)(n1−−√μ¯1,n2−−√μ¯2,⋯,nG−−−√μ¯G)T=1NXT(1ng−−√1g)N×G(I−1NKKT)(1ng−−√1g)TN×GX=1NXT(1ng−−√1g)N×GC(1ng−−√1g)TN×GX

Claim that C=H~TH~, where H~∈RG−1×G. That is to say

C=(I−1NKKT)=H~H~T

so (K,H~T) is an orthogonal matrix. From the theory of orthogonal contrasts for unbalanced data , we have the G−1 orthogonal contrasts have the following form:

δr=nr+1−−−−√(∑h=1rnh(μ¯h−μ¯r+1))

Denoted by hr the rth row of H~. Then from the definition of orthogonal contrasts, for some constant Cr,

XT(1ng−−√)hTr=Crδr

which can be rewritten as

∑j=1Ghrhnj−−√μ¯j=Crnr+1−−−−√⎛⎝∑j=1rnj(μ¯j−μ¯r+1)⎞⎠

Then

hrjnj−−√hrr+1nr+1−−−−√hri=Crnr+1−−−−√nj=−Crnr+1−−−−√∑t=1rnt=0forj=1,2,⋯,r

which gives

hrjhrr+1hri=Crnr+1nj−−−−−−√=−Cr∑t=1rnt=0forj=1,2,⋯,r

To making ∥hr∥2=1, set

C2r⎛⎝∑i=1rnr+1ni+⎛⎝∑j=1rnj⎞⎠2⎞⎠=1

Cr=1∑ri=1ni∑r+1j=1nj

Now

XT(1ng−−√1g)hr=nr+1−−−−√∑ri=1ni∑r+1j=1nj(∑h=1rnh(μ¯h−μ¯r+1))

So

Σ^b=1NXT(1ng−−√1g)N×GC(1ng−−√1g)TN×GX=1NXT(1ng−−√1g)N×GH~TH~(1ng−−√1g)TN×GX=ΔΔT

where Δ=1N√XT(1ng√1g)N×GH~T
and

Δr=nr+1−−−−√N−−√∑ri=1ni∑r+1j=1nj(∑h=1rnh(μ¯h−μ¯r+1))