Skip to content
ML Academy

Introduced on day 31

Forward pass

The computation from input to output (and loss) through the layers in order, storing the intermediate values that the backward pass will need.

In detail

For a dense network it alternates matrix products and activations. At inference only the forward pass runs; in training the intermediates are kept so backpropagation can apply the chain rule, which is why training needs more memory than inference.