Skip to content
ML Academy

Introduced on day 37

Recurrent network (RNN)

A network that reads a sequence one step at a time, updating a fixed-size hidden state with the same weights at every step.

In detail

The update ht=tanh⁡(Wxxt+Whht−1+b)h_t = \tanh(W_x x_t + W_h h_{t-1} + b) makes the network a deep graph with tied weights when unrolled. LSTM and GRU cells add gates to control what the state keeps. Recurrence handles variable-length input but computes steps in order, which limits parallelism, and everything the future needs must pass through the state.