Week 45, Convolutional Neural Networks (CCNs) and Recurrent Neural Networks (RNNs)

 

 

 

What kinds of behaviour can RNNs exhibit?

  1. They can oscillate.
  2. They can settle to point attractors.
  3. They can behave chaotically.
  4. RNNs could potentially learn to implement lots of small programs that each capture a nugget of knowledge and run in parallel, interacting to produce very complicated effects.

But the extensive computational needs of RNNs makes them very hard to train.