Choosing the next token from the probability distribution, rather than always taking the most likely one.
Broad technical consensus on what this term means. Disagreements that invoke it are about the world, not the word.
Once the output head has produced probabilities, something has to pick. Always taking the highest is available and produces flat, repetitive text, so most systems draw from the distribution under controls such as temperature, top-k, or nucleus sampling.
This is the step that makes a model non-deterministic, and it sits entirely outside the network. The same weights and the same prompt can yield different text because of a choice made after all the arithmetic is finished.