Writing a code which implements a feed-forward neural network

Last week we discussed the basics of neural networks and deep learning and the basics of automatic differentiation. We looked also at examples on how compute the parameters of a simple network with scalar inputs and ouputs and no or just one hidden layers.

We ended our discussions with the derivation of the equations for a neural network with one hidden layers and two input variables and two hidden nodes but only one output node.