Temperature
Also known as LLM temperature, Sampling temperature
Definition
Temperature is a generation parameter that changes how a language model samples from the probability distribution of possible next tokens. Higher values generally allow more variation, while lower values generally favor higher-probability choices.
What temperature changes
At each generation step, a model assigns probabilities to candidate next tokens. Temperature reshapes that distribution before sampling. A lower setting concentrates probability around likely choices, which can make output more repeatable. A higher setting spreads probability across more choices, which can increase variety and also increase the chance of an unsuitable continuation.
Temperature does not add knowledge or make a model reason more deeply. It changes the selection behavior of the generation process. Provider implementations, supported ranges, defaults, and interactions with other sampling settings can differ, so the configured value should be recorded with the provider and model version.
Choose it by task
A deterministic extraction task may benefit from a low temperature, especially when the output is validated against a schema. A brainstorming task may benefit from more variation. These are starting points rather than guarantees. Run a representative evaluation dataset and inspect failures, repetition, usefulness, and consistency.
Keep other variables stable while comparing settings. Record prompts, retrieved context, model, route, and generation parameters. If a production workflow retries a request, measure the final user outcome and the additional tokens and latency rather than only the first response.
How this relates to Weave
Weave evaluations can compare temperature settings on the same task set and model route. Record the parameter with each generation so quality, latency, and cost comparisons remain reproducible.
Explore Router