Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression

 

 

 

Plans for week 35

The main topics are:

  1. Brief repetition from last week
  2. Derivation of the equations for ordinary least squares
  3. Discussion on how to prepare data and examples of applications of linear regression
  4. Material for the lecture on Thursday: Mathematical interpretations of linear regression
  5. Thursday: Ridge and Lasso regression and Singular Value Decomposition
  6. Video of lecture
  7. Whiteboard notes

Reading recommendations:

  1. See lecture notes for week 35 at https://compphysics.github.io/MachineLearning/doc/web/course.html
  2. Goodfellow, Bengio and Courville, Deep Learning, chapter 2 on linear algebra and sections 3.1-3.10 on elements of statistics (background)
  3. Hastie, Tibshirani and Friedman, The elements of statistical learning, sections 3.1-3.4 (on relevance for the discussion of linear regression).