The hope is that the training of the autoencoder can unravel some useful properties of the function \( f \). They are often trained with only single-layer neural networks (although deep networks can improve the training) and are essentially given by feed forward neural networks.