Skip to content
ML Academy

Introduced on day 34

Learning-rate schedule

A rule that changes the learning rate over training, typically a short warmup from a small value followed by a decay such as cosine.

In detail

Warmup avoids unstable early steps; decay lets the model settle into a minimum with small steps. The schedule is a hyperparameter tied to total step count, so changing the number of epochs means changing the schedule too.