Week 46: Decision Trees, Ensemble methods and Random Forests
Contents
Plan for week 46
Reading recommendations
TensorFlow examples
From NNs and CNNs to recurrent neural networks (RNNs)
What is a recurrent NN?
Why RNNs?
Feedback connections
Vanishing gradients
Recurrent neural networks (RNNs): Overarching view
Sequential data only?
Differential equations
A simple regression example using TensorFlow with Keras
Corresponding example using PyTorch
RNNs
What kinds of behaviour can RNNs exhibit?
Basic layout, "Figures from Sebastian Rashcka et al, Machine learning with Sickit-Learn and PyTorch":"https://sebastianraschka.com/blog/2022/ml-pytorch-book.html"
Solving differential equations with RNNs
Two first-order differential equations
Velocity only
Linking with RNNs
Minor rewrite
RNNs in more detail
RNNs in more detail, part 2
RNNs in more detail, part 3
RNNs in more detail, part 4
RNNs in more detail, part 5
RNNs in more detail, part 6
RNNs in more detail, part 7
Backpropagation through time
The backward pass is linear
The problem of exploding or vanishing gradients
Mathematical setup
Back propagation in time through figures, part 1
Back propagation in time, part 2
Back propagation in time, part 3
Back propagation in time, part 4
Back propagation in time in equations
Chain rule again
Gradients of loss functions
Summary of RNNs
Summary of a typical RNN
Four effective ways to learn an RNN
The mathematics of RNNs, the basic architecture
Plan for week 46
Intro to and mathematics of Recurrent Neural Networks (RNNs)
Lecture notes at
https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week46/ipynb/week46.ipynb
Video of lecture at
https://youtu.be/WNWKaBbqOpg
Whiteboard notes at
https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2025/FYSSTKweek46.pdf
Work on and discussions of project 3, deadline December 15 at midnight. Project 3 can be found at
https://github.com/CompPhysics/MachineLearning/tree/master/doc/Projects/2025/Project3
.
Note that possible minor revisions may be done before the lab sessions start on November 11
.
«
1
2
3
4
5
6
7
8
9
10
11
...
44
»