Introduced on day 42
Temperature (sampling)
A positive number that divides the model's scores before the softmax; values below 1 sharpen the next-token distribution and values above 1 flatten it.
In detail
Temperature changes how often lower-ranked tokens are drawn, not which tokens the model prefers. As it approaches zero, decoding approaches always choosing the highest-scoring token. It does not add facts, and on hosted systems a setting of zero still may not give identical outputs across runs.