Temperature: why answers vary

When the model predicts the next token it does not find just one candidate. It produces a list of possible next tokens, each with a probability (how likely it thinks that token is). After "The sky is", "blue" might be very likely, "clear" somewhat likely, "falling" unlikely.

The model then picks one. If it always picked the single most likely token, answers would be repetitive and dull. So by default there is a bit of randomness, which is why asking the same question twice gives different wording.

Temperature is a setting that controls this randomness:

• Low temperature (near 0): the model almost always takes the most likely token. Answers are more focused and repeatable. Good for extracting data, classifying, or answering from a document.
• Higher temperature: less likely tokens get picked more often. Answers are more varied and creative. Good for brainstorming names or drafting several options.

Some things to know in 2026:

• Many newer models, especially the "reasoning" models covered later, ignore or reject a custom temperature, and their providers recommend leaving the default alone. Often there is simply no temperature box.
• Even at temperature 0 the output is not guaranteed to be identical every time.
• Temperature changes style and variety, not knowledge. Turning it down does not make the model more truthful.

If you need consistent results, rely first on clear instructions and examples of the desired output (covered next), and on asking for a fixed format. Use temperature as a fine-tuning knob only where your model supports it.