Setting up the back propagation algorithm and algorithm for a feed forward NN, initalizations
- Set up your inputs and outputs (scalars, vectors, matrices or higher-order arrays)
- Define the number of hidden layers and hidden nodes
- Define activation functions for hidden layers and output layers
- Define optimizer (plan learning rate, momentum, ADAgrad, RMSprop, ADAM etc) and array of initial learning rates
- Define cost function and possible regularization terms with hyperparameters
- Initialize weights and biases
- Fix number of iterations for the feed forward part and back propagation part