Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression

 

 

 

Plans for week 35

The main topics are:

  1. Brief repetition from last week
  2. Discussions of the equations for ordinary least squares (OLS)
  3. Discussion on how to prepare data and examples of applications of linear regression
  4. Mathematical interpretations of OLS
  5. Introduction of Ridge and Lasso regression

Reading recommendations:

  1. These lecture notes
  2. Video of lecture at https://youtu.be/2mvizAQFST8
  3. Whiteboard notes at https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2025/FYSSTKweek35.pdf
  4. Goodfellow, Bengio and Courville, Deep Learning, chapter 2 on linear algebra
  5. Raschka et al on preprocessing of data, relevant for exercise 3 this week, see chapter 4.
  6. For exercise 1 of week 35, the book by A. Aldo Faisal, Cheng Soon Ong, and Marc Peter Deisenroth on the Mathematics of Machine Learning, may be very relevant. In particular chapter 5 at URL"https://mml-book.github.io/" (section 5.5 on derivatives) is very useful for exercise 1 this coming week.