Week 38: Logistic Regression and Optimization
Contents
Plans for week 38
Material from last week and relevant for the first project
Various steps in cross-validation
How to set up the cross-validation for Ridge and/or Lasso
Cross-validation in brief
Code Example for Cross-validation and \( k \)-fold Cross-validation
Material for lecture Thursday September 21
Logistic Regression
Classification problems
Optimization and Deep learning
Basics
Linear classifier
Some selected properties
Simple example
Plotting the mean value for each group
The logistic function
Examples of likelihood functions used in logistic regression and nueral networks
Two parameters
Maximum likelihood
The cost function rewritten
Minimizing the cross entropy
A more compact expression
Extending to more predictors
Including more classes
More classes
Friday September 23
Searching for Optimal Regularization Parameters \( \lambda \)
Grid Search
Randomized Grid Search
Wisconsin Cancer Data
Using the correlation matrix
Discussing the correlation data
Other measures in classification studies: Cancer Data again
Optimization, the central part of any Machine Learning algortithm
Revisiting our Logistic Regression case
The equations to solve
Solving using Newton-Raphson's method
Brief reminder on Newton-Raphson's method
The equations
Simple geometric interpretation
Extending to more than one variable
Steepest descent
More on Steepest descent
The ideal
The sensitiveness of the gradient descent
Convex functions
Convex function
Conditions on convex functions
More on convex functions
Some simple problems
Revisiting our first homework
Gradient descent example
The derivative of the cost/loss function
The Hessian matrix
Simple program
Gradient Descent Example
And a corresponding example using
scikit-learn
Gradient descent and Ridge
The Hessian matrix for Ridge Regression
Program example for gradient descent with Ridge Regression
Using gradient descent methods, limitations
Challenge yourself the coming weekend
Material from last week and relevant for the first project
«
1
2
3
4
5
6
7
8
9
10
11
12
...
63
»