Machine Learning Foundations

1.  when can machines learn?

1.1 The Learning Problem

1.1.1 what is machine learning

learning: acquiring skill

  with experience accumulated from observations

  observations --> learning --> skill

machine learning: acquiring skill <-> improving some performance measure

  with experience accumulated/computed from data

  data --> machine learning --> skill (improved performance measure)

ML: an alternative route to build complicated systems

Some Use Scenarios
• when human cannot program the system manually
  —navigating on Mars
• when human cannot ‘define the solution’ easily
  —speech/visual recognition
• when needing rapid decisions that humans cannot do
  —high-frequency trading
• when needing to be user-oriented in a massive scale
  —consumer-targeted marketing

Key Essence of Machine Learning

1 exists some ‘underlying pattern’ to be learned
  —so ‘performance measure’ can be improved
2 but no programmable (easy) definition
  —so ‘ML’ is needed
3 somehow there is data about the pattern
  —so ML has some ‘inputs’ to learn from

1.1.2 Components of Machine Learning
A takes D and H to get g

1.1.3 Machine Learning and Other Fields

difficult to distinguish ML and DM in reality

Machine Learning: use data to compute hypothesis g that approximates target f
Data Mining: use (huge) data to find property that is interesting

• if ‘interesting property’ same as ‘hypothesis that approximate target’
  —ML = DM (usually what KDDCup does)
• if ‘interesting property’ related to ‘hypothesis that approximate target’
  —DM can help ML, and vice versa (often, but not always)
• traditional DM also focuses on efficient computation in large database

ML is one possible route to realize AI

Machine Learning: use data to compute hypothesis g that approximates target f
Artificial Intelligence: compute something that shows intelligent behavior

• g ≈ f is something that shows intelligent behavior
  —ML can realize AI, among other routes
• e.g. chess playing
  • traditional AI: game tree
  • ML for AI: ‘learning from board data’

statistics: many useful tools for ML

Machine Learning: use data to compute hypothesis g that approximates target f
Statistics: use data to make inference about an unknown process
• g is an inference outcome; f is something unknown
  —statistics can be used to achieve ML
• traditional statistics also focus on provable results with math assumptions, and care less about computation