• Exercise on writing your own stochastic gradient and gradient descent codes. This exercise continues from the previous week but now with inclusion of automatic differentiation
• Discussion of project 2
• See video on automatic differentiation from last year. This video will be updated before Tuesday.

• Building our own Feed-forward Neural Network and discussion of project 2
• Readings and Videos:
• These lecture notes
• Video of lecture
• Whiteboard notes
• Aurelien Geron's chapters 10-11
• For a more in depth discussion on neural networks we recommend Goodfellow et al chapters 6 and 7.
• Neural Networks demystified
• Building Neural Networks from scratch
• Video on Neural Networks
• Video on the back propagation algorithm

I also recommend Michael Nielsen's intuitive approach to the neural networks and the universal approximation theorem, see the slides at http://neuralnetworksanddeeplearning.com/chap4.html.