Introduced on day 40
Frozen layers
Layers whose weights are excluded from updates, so they keep their pretrained values and carry no gradients or optimizer states.
In detail
Their weights still occupy memory and their forward pass still costs compute. If nothing below a frozen layer is trainable, its activations need not be stored for backprop. Typical choices run from a fully frozen backbone with a new head to unfreezing the top blocks gradually with smaller learning rates for lower layers.