Week 40: Gradient descent methods (continued) and start Neural networks
Contents
Lecture Monday September 29, 2025
Suggested readings and videos
Lab sessions Tuesday and Wednesday
Logistic Regression, from last week
Classification problems
Optimization and Deep learning
Basics
Two parameters
Maximum likelihood
The cost function rewritten
Minimizing the cross entropy
A more compact expression
Extending to more predictors
Including more classes
More classes
Optimization, the central part of any Machine Learning algortithm
Revisiting our Logistic Regression case
The equations to solve
Solving using Newton-Raphson's method
Example code for Logistic Regression
Synthetic data generation
Using
Scikit-learn
Using the correlation matrix
Discussing the correlation data
Other measures in classification studies
Introduction to Neural networks
Artificial neurons
Neural network types
Feed-forward neural networks
Convolutional Neural Network
Recurrent neural networks
Other types of networks
Multilayer perceptrons
Why multilayer perceptrons?
Illustration of a single perceptron model and a multi-perceptron model
Examples of XOR, OR and AND gates
Does Logistic Regression do a better Job?
Adding Neural Networks
Mathematical model
Mathematical model
Mathematical model
Mathematical model
Mathematical model
Matrix-vector notation
Matrix-vector notation and activation
Activation functions
Activation functions, Logistic and Hyperbolic ones
Relevance
Suggested readings and videos
The lecture notes for week 40 (these notes)
For neural networks we recommend Goodfellow et al chapter 6 and Raschka et al chapter 2 (contains also material about gradient descent) and chapter 11 (we will use this next week)
Neural Networks demystified at
https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&ab_channel=WelchLabs
Building Neural Networks from scratch at URL:https://www.youtube.com/watch?v=Wo5dMEP_BbI&list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&ab_channel=sentdex"
«
1
2
3
4
5
6
7
8
9
10
11
12
...
48
»