Introduced on day 33
Vanishing gradient
Gradients that shrink toward zero as they are backpropagated through many layers, so early layers learn very slowly or not at all.
In detail
Typical causes are saturating activations (a sigmoid slope is at most 0.25), weights that are too small, and depth without residual paths. Per-layer gradient norms reveal it as a steady decline toward the input. Remedies change the architecture or initialization: ReLU-family activations, suitable initialization, normalization, residual connections. The mirror image is the exploding gradient, handled by clipping.