Skip to content
ML Academy

Introduced on day 36

Parameter sharing

Reusing the same weights at every position (convolution) or every time step (recurrent networks), so the parameter count does not grow with input length.

In detail

A conv filter of 5 x 5 has 25 weights whether the image is 28 x 28 or 224 x 224. It builds in the assumption that a pattern means the same thing wherever it occurs. A recurrent cell shares weights over time in the same spirit (Day 37). The saving is in parameters; compute and activation memory still scale with input size.