Lecture4-4Connection to Real Learning

Multiple h

Real Learning like PLA, what about when getting all green(right)?

Bad Sample:

Ein and Eout far away – can get worse results.

Eout=12, but getting all heads(Ein=0)!

Bad Data for one h

Eout(h) and Ein(h) far away: e.g., Eout(h) big( far from f ), but Ein(h) small(correct on most examples).

Bad Data for many h

Bad data for many h
⟺ no ‘freedom of choice’ by 
⟺ there exists some h such that Eout(h) and Ein(h) are far away

Bound of bad data

PD(BAD D)=PD(BAD D for h1 or BAD D for h2 or ...BAD D for hM)≤PD(BAD D for h1)+PD(BAD D for h2)+...+PD(BAD D for hM)≤2Mexp(−2ϵ2N)

• Finite-bin version of Hoeffding, valid for all M,N,ϵ
• does not depend on any Eout(hm), no need to know Eout(hm) —– f and P can stay unknown
• Ein(g)=Eout(g) is PAC, regardless of 

Most reasonable : pick the hm with lowest Ein(hm) as g.

Flow Chart 4

Summary