Skip to content
ML Academy

Introduced on day 33

Computational graph

The network and loss written as a graph of simple operations (matrix product, add, ReLU, sigmoid, loss) whose local derivatives are combined by the chain rule.

In detail

The forward pass evaluates the graph from inputs to loss. The backward pass walks it in reverse, multiplying each operation's local derivative by the gradient arriving from its output. Frameworks build this graph automatically (automatic differentiation), which is why custom operations need correct local derivatives.