Introduced on day 42
Top-p (nucleus) sampling
A decoding rule that samples only from the smallest set of most likely tokens whose probabilities add up to p, then renormalises.
In detail
The size of the kept set adapts to the distribution: a confident step may keep one token, an uncertain step many. It removes the unlikely tail that causes incoherent text while keeping variety. Top-k is the fixed-size alternative that always keeps the k most likely tokens.