• Exercise on writing your own stochastic gradient and gradient descent codes. This exercise continues next week with studies of automatic differentiation
• One lecture at the beginning of each session on the material from weeks 39 and 40 and how to write your own gradient descent code
• Discussion of project 2
• Your task before the sessions: revisit the material from weeks 39 and 40 and in particular the material from week 40 on stochastic gradient descent

• Neural Networks, setting up the basic steps, from the simple perceptron model to the multi-layer perceptron model.
• Building our own Feed-forward Neural Network
• Video of lecture notes
• Whiteboard notes
• Readings and Videos:
• These lecture notes
• For neural networks we recommend Goodfellow et al chapter 6.
• Neural Networks demystified
• Building Neural Networks from scratch
• Video on Neural Networks
• Video on the back propagation algorithm

I also recommend Michael Nielsen's intuitive approach to the neural networks and the universal approximation theorem, see the slides at http://neuralnetworksanddeeplearning.com/chap4.html.