Week 47: Recurrent neural networks and Autoencoders

 

 

 

Core LSTM Equations

$$ \begin{align*} f_t &= \sigma(W_f [h_{t-1}, x_t] + b_f), \\ i_t &= \sigma(W_i [h_{t-1}, x_t] + b_i), \\ \tilde{C}_t &= \tanh(W_C [h_{t-1}, x_t] + b_C), \\ C_t &= f_t \odot C_{t-1} + i_t \odot \tilde{C}_t, \\ o_t &= \sigma(W_o [h_{t-1}, x_t] + b_o), \\ h_t &= o_t \odot \tanh(C_t). \end{align*} $$
  1. \( \sigma \) is the sigmoid function, \( \odot \) is elementwise product [oai_citation:4‡jaketae.github.io](https://jaketae.github.io/study/dissecting-lstm/#:~:text=%5C%5B%5Cbegin,align).
  2. These equations define how LSTM retains/updates memory and produces outputs.