Skip to content
ML Academy

Introduced on day 44

Adapter (parameter-efficient)

A small trainable module or low-rank update attached to a frozen pretrained model so that a task can be learned without changing the base weights.

In detail

Bottleneck adapters in the Houlsby sense are extra layers inserted inside each transformer block and add computation at inference; LoRA adapters are low-rank updates that can be merged into the base weights. One base model can serve several tasks by loading different adapters, which makes adapter files part of the release and versioning story.