ML Table of Contents

Posts I wrote for an upper-division seminar course on Machine Learning. I need to polish them when I have a chance, especially those that came late in the semester. There are also significant additional topics that should be included in the future — e.g. Generative models and topics in Unsupervised Learning; or VC dimension.

---

Linear models for Binary classification and Regression

• • Binary classification and Half-space models
  • Implementation: Iris flower data set
  • Logistic regression
  • Linear regression
  • Implementing linear regression

Selecting a model with Empirical Risk Minimization, Hypothesis classes

• • Loss functions: Empirical Error (Risk) vs Generalization Error
  • Learning paradigm: Empirical Risk Minimization (ERM) with Bias
  • Bias-Complexity (Bias-Variance) tradeoff
  • PAC learnability and Weak learnability (first half of post)

Decision stumps and Boosting

• • Decision stumps (second half of post)
  • Boosting (AdaBoost) on Decision stumps, with an application

Learning paradigm: SGD

• • Gradient Descent and Stochastic Gradient Descent (SGD)
  • SGD for Logistic Regression

More powerful ML models

• • k-Nearest Neighbors (kNN)
  • Decision Trees
  • Intro to Neural Networks
  • Backpropagation and training Neural Networks