Introduced on day 33
Learning rate
The scale factor on the gradient (or on the optimizer's step) that sets how far parameters move in one update; the most influential training setting.
In detail
Too large and the loss spikes or turns nan; too small and training crawls or stalls. Good values differ by optimizer and depend on batch size and normalization. Schedules (warmup, decay) vary it over training, and a range test that raises it over a few hundred steps gives a starting estimate.