Skip to content
ML Academy

Introduced on day 31

Batch dimension

The leading tensor axis that indexes examples; it passes unchanged through every layer, and no parameter shape depends on it.

In detail

With the rows-are-examples convention the batch is axis 0. Activation memory grows with the batch size; the model's parameter count does not. A layer should compute each example independently of the others (except normalization layers that use batch statistics), which is why mixing across the batch axis is a bug to test for.