Quantization

　　 For one-dimensional random variable, we use scalar quantization to quantize it. And for random vector, which can be regarded as combination of n random variables, we use vector quantization to quantize it. Notice that the random variables which combine random vector should be independently. A scaler partitions the set R of real number into M subset R1,R2,...,RM, called quantization region. Each quantization region is an interval. Each region Rj is represented by a representation point aj∈R, e.t, any at region Rj with u∈Rj will be quantized to aj.
　　Initially assume that the region are intervals, order as in Figure, with R1=(−∞,b1],R2=(b1,b2],…,RM=(bM−1,∞). Each region Rj is begin with bj−1 and end with bj.

　　For a given value of M, how can we choose the regions and representation points to minimize mean squared error? The question should be considered as follows:
(1) Given a set of representation point {aj}, how do we choose the interval {Rj}?
(2) Given a set of interval {Rj}, how do we choose the representation points {aj}?
　　For the first question, provided that a set of representation point {aj} is given，given any u∈Rj, the squared error to aj is (u−aj)2. The goal in this situation is to minimize the squared error (u−aj)2. For example, if u is between aj and aj+1, the u should be mapped into closer of two. Thus boundary bj between Rj and Rj+1 must lie halfway between the representation points aj and aj+1 as follow picture.



That is bj=aj+aj+12.
　　 For the second question, given a set of quantization region {Rj}, given any u∈{Rj} must have one representation value. We assume that sets of sampling value in each sampling time is {Uk}. Obviously, {Uk} is a random variable. Assume that random variable {Uk} are iid with the pdf fuu. For a given set of point {aj}, each sampling value {uk} map into aj. The mean squared error is then

MSE=∫+∞−∞fu(u)(u−V(u))2du=∑j=1M∫Rjfu(u)(u−aj)2du

We can minimize that formula over {aj} by separate minimizations over each region Rj(Remember that the region here are fixed). In other words, the integral over {aj} in real number R is equal to the sum of each integral in different interval. Let fj(u) denote that conditional pdf of u given that u∈Rj, i.e,

fj(u)=⎧⎩⎨⎪⎪fu(u)Qju∈Rj0otherwise

where Qj=Pr{u∈Rj}. Then, for the interval Rj, we can get

∫Rjfu(u)(u−aj)2du=Qj∫Rjfj(u)(u−aj)2du

Now that formula is minimized by choosing aj to be the mean of (u−aj)2 over u∈Rj. To see this, notice that for any random variable Y and fixed number a, we have

E[(Y−a)2]=E(Y2+a2−2aY)=E(Y2)−2aE(Y)+a2

which is minimized over a when a=Y¯¯¯. For understanding better, E[(Y−a)2] is variance when a=E(Y).
Proof:

E[(Y−a)2]=∫+∞−∞(y−a)2f(y)dy

obtaining partial derivative of E[(Y−a)2] for a, and letting it become 0, we have

∂E[(Y−a)2]∂a=−2∫+∞−∞(y−a)f(y)dy=0

Then

a=∫+∞−∞yf(y)dy=Y¯¯¯

Above discussion, we found a set of conditions that end points {bj} and a set of quantization points {aj} satisfy that each bj must be midpoint between aj and aj+1, and each aj must be the mean of an random variable uj with pdf fj(u) on the constraint u∈Rj. In other words, aj must be the conditional mean of u conditional on u∈Rj