Start from modern definition of Conditional Expectation

Definitions

The conditional probability of the event A∈F under the conditional of η=y P(A|η=y) is E(IA|η=y).

Let (ξ,η) be a pair of random variables whose joint distribution has a joint density function fξη(x,y) iff

P((ξ,η)∈A)=∫Afξη(x,y)dxdy∀A∈B(R2)

And the density function of ξ denoted by fξ(x).

The density of a conditional probability distribution of ξ with respect to η=y denoted by fξ|η(x|y) is defined as

P(ξ∈C|η=y)=∫Cfξ|η(x|y)dx,∀C∈B(R)

Some useful formulas

It is clear that

P(A∩{η∈B})=∫BP(A|η=y)dPη(y)∀B∈B(R¯)

Proof:

P(A∩{η∈B})=∫A∩{η∈B}dP=∫{η∈B}IAdP=∫{η∈B}E[IA|η]dP=∫BE[IA|η=y]dPη(y)=∫BP(A|η=y)dPη(y)

---

fξ|η(x|y)=˙fξη(x,y)fη(y)

Proof:
∀B∈B(R) and ∀C∈B(R):

∫B∫Cfξη(x,y)fη(y)dxdPη(y)=∫C∫Bfξη(x,y)fη(y)dPη(y)dx=∫C∫Bfξη(x,y)dydx=P({ξ∈C}∩{η∈B})=∫BP(ξ∈C|η=y)dPη(y)

This implies that ∀C∈B(R):

∫Cfξη(x,y)fη(y)dx=˙P(ξ∈C|η=y)

This complete the proof.

---

If Eg(ξ)<∞,then

E(g(ξ)|η=y)=˙∫Rg(x)fξ|η(x|y)dx

Specially,

E(ξ|η=y)=˙∫Rxfξ|η(x|y)dx

Proof:
∀C∈B(R)

∫C∫Rg(x)fξ|η(x|y)dxdPη(y)=∫R∫Cg(x)fξη(x,y)dydx=∫η∈Cg(ξ)dP=∫η∈CE[g(ξ)|η]dP=∫CE[g(ξ)|η=y]dPη(y)

It follows that

E(g(ξ)|η=y)=˙∫Rg(x)fξ|η(x|y)dx

This complete the proof.

The following item is the general form of the above.

---

Suppose that the random variables (X,Y) are such that there is a regular distribution P(Y∈B|X=x).If E|f(Y)| exists,then

E[f(Y)|X=x]=˙∫Rf(y)P(dy|X=x)

Proof:
Case 1:
∀A∈B(R),let f=IA, then

∫Rf(y)P(dy|X=x)=˙P(Y∈A|X=x)=E[f(Y)|X=x]

Case 2:
It follows Case 1 that the conclusion holds if the f is any given simple function.
Case 3:
If f is nonnegative,there is a sequence of simple functions {fn},such that

fn↗f

E[f(Y)|X=x]=˙limn→∞E[fn(Y)|X=x]=˙limn→∞∫Rfn(y)P(dx|X=x)=˙∫Rf(y)P(dy|X=x)

Case 4:
Due to f=f+−f−,apply case 3 to f+ and f− respectively.
This complete the proof.
continue…