Token budget
Also known as LLM token budget, AI token budget
Definition
A token budget is a configured limit or allowance for the tokens an AI system may process or generate during a request, task, or accounting period. The exact scope can refer to output length, context capacity, spending, or a workflow's total usage.
Clarify what the budget covers
An output budget limits generated tokens. A context budget limits the material a model can consider. A spend budget limits cost over time. An agent task budget may include several model calls and tool steps.
These limits can interact. A request with a long context leaves less room for output when both share a context window. A short per-call limit can force an agent into more calls to complete one task.
Set a budget around a task
Suppose an illustrative coding agent has enough output allowance to produce a patch but not enough to run its verification loop. The request may stop before the useful outcome. Increasing the allowance could help, or a better prompt and smaller tool results could reduce the required output.
Track completions, truncations, retries, latency, and cost. Do not judge a budget by token savings alone.
Make the policy visible
Document whether the budget applies to one request, one task, one user, or an organization. Include model and provider rules, since token accounting can differ. Revisit the budget when task mix, context size, or model behavior changes.
How this relates to Weave
Weave's Token Intelligence helps teams inspect token usage and cost by tool, user, or workflow. Use budgets as guardrails and compare them with completed outcomes. A lower token allowance can reduce spending while increasing retries or incomplete tasks.
Explore Token intelligence