Setting up the back propagation algorithm and algorithm for a feed forward NN, initalizations

  1. Set up your inputs and outputs (scalars, vectors, matrices or higher-order arrays)
  2. Define the number of hidden layers and hidden nodes
  3. Define activation functions for hidden layers and output layers
  4. Define optimizer (plan learning rate, momentum, ADAgrad, RMSprop, ADAM etc) and array of initial learning rates
  5. Define cost function and possible regularization terms with hyperparameters
  6. Initialize weights and biases
  7. Fix number of iterations for the feed forward part and back propagation part