Week 46: Decision Trees, Ensemble methods and Random Forests
Contents
Plan for week 46
Reading recommendations
TensorFlow examples
From NNs and CNNs to recurrent neural networks (RNNs)
What is a recurrent NN?
Why RNNs?
Feedback connections
Vanishing gradients
Recurrent neural networks (RNNs): Overarching view
Sequential data only?
Differential equations
A simple regression example using TensorFlow with Keras
Corresponding example using PyTorch
RNNs
What kinds of behaviour can RNNs exhibit?
Basic layout, "Figures from Sebastian Rashcka et al, Machine learning with Sickit-Learn and PyTorch":"https://sebastianraschka.com/blog/2022/ml-pytorch-book.html"
Solving differential equations with RNNs
Two first-order differential equations
Velocity only
Linking with RNNs
Minor rewrite
RNNs in more detail
RNNs in more detail, part 2
RNNs in more detail, part 3
RNNs in more detail, part 4
RNNs in more detail, part 5
RNNs in more detail, part 6
RNNs in more detail, part 7
Backpropagation through time
The backward pass is linear
The problem of exploding or vanishing gradients
Mathematical setup
Back propagation in time through figures, part 1
Back propagation in time, part 2
Back propagation in time, part 3
Back propagation in time, part 4
Back propagation in time in equations
Chain rule again
Gradients of loss functions
Summary of RNNs
Summary of a typical RNN
Four effective ways to learn an RNN
The mathematics of RNNs, the basic architecture
Reading recommendations
For RNNs, see Goodfellow et al chapter 10, see
https://www.deeplearningbook.org/contents/rnn.html
.
Reading suggestions for implementation of RNNs in PyTorch: see Rashcka et al.'s chapter 15 and GitHub site at
https://github.com/rasbt/machine-learning-book/tree/main/ch15
.
«
1
2
3
4
5
6
7
8
9
10
11
12
...
44
»