Introduced on day 31
Broadcasting
The rule by which numpy and PyTorch combine arrays of different shapes by stretching size-1 or missing leading axes, so a bias vector is added to every row of a batch.
In detail
Adding a bias of shape (units,) to activations of shape (batch, units) repeats the bias for each row. The same rule can silently turn a shape mistake into a wrong but valid computation, for example a (B,) vector subtracted from a (B, 1) column producing a (B, B) matrix, so assert shapes before the loss.