Input token
Also known as Prompt token, Input tokens
Definition
An input token is a unit of text or other encoded content sent to a language model before generation. The prompt, system instructions, conversation history, retrieved passages, and tool results can all contribute input tokens.
What counts
Input tokens include more than the visible user message. System prompts, previous turns, tool definitions, retrieved documents, and serialized data are part of the model input. Different tokenizers split the same text differently, so counts are model specific.
Why input size matters
More context can raise cost and processing time while leaving less room for the response. Large context can also distract a model when irrelevant material is included. Measure the context that actually improves task success before keeping it by default.
Improve the signal
Remove duplicated instructions, summarize old turns, retrieve narrower passages, and use caching for stable prefixes when supported. Weave connects input tokens to latency, route, and evaluation results so teams can optimize context with evidence.
Inspect both average and high percentile input size. A small number of very large requests can dominate spend and cause timeouts even when the average prompt looks healthy.
How this relates to Weave
Weave shows input token volume by model, route, feature, and request type. That helps teams find repeated context, retrieval growth, and prompts that increase cost without improving completed work.
Explore Token intelligence