Introduced on day 37
Hidden state
The vector a recurrent network carries from step to step; a learned, fixed-size, lossy summary of the sequence so far.
In detail
It is an activation, recomputed for every sequence, not a set of weights. An output layer reads it to produce a forecast or label. Because the same update rule overwrites it at every step, what it keeps is what training found useful for the loss.