Introduced on day 34
Dropout
A regularizer that zeroes each unit at random with probability p during training and is switched off at evaluation.
In detail
Modern frameworks use inverted dropout: the surviving activations are scaled by 1/(1-p) during training so that evaluation needs no correction. Forgetting to switch to evaluation mode makes predictions random. Too high a rate causes underfitting. Averaging several dropout passes at inference (Monte Carlo dropout) is a separate technique with its own validation needs.