Prompt caching
Also known as Prompt prefix caching
Definition
Prompt caching reuses processing associated with previously supplied prompt content, often a matching prefix, to reduce repeated input work. It differs from response caching, which returns a stored answer instead of generating a new one.
Repeated context creates an opportunity
A coding agent may send the same instructions and tool definitions on many consecutive turns. Reprocessing that stable material from scratch can duplicate work. A prompt cache can reuse eligible earlier processing while the model still generates a fresh response.
Provider rules determine what qualifies. Matching requirements, minimum lengths, retention periods, pricing, and explicit cache controls can differ. Treat these as implementation details to verify, not assumptions that transfer automatically between services.
Request order can affect reuse
In an illustrative workflow, stable instructions precede the changing user request. This arrangement can preserve a reusable prefix. Inserting a changing timestamp before the stable instructions may prevent a prefix match, depending on the cache implementation.
That does not mean prompts should be reordered without testing. Instruction placement and application semantics still matter. Optimize reuse while preserving the behavior the request is meant to produce.
Count the right kind of savings
Cache writes, cache reads, uncached input, and generated output may have different accounting rules. A high cache-hit percentage alone does not describe the full bill.
Track the relevant usage categories and compare complete tasks. If the workflow switches models, creates more attempts, or generates longer responses, those changes can offset cheaper input processing. Cached input also remains part of the request's context; caching should not be mistaken for an unlimited context window.
How this relates to Weave
Caching changes the economics of a routed agent workflow. Compare Weave Router using complete task costs and realistic repeated context, rather than multiplying every input token by an uncached rate. A change in model or provider may change which cached prefixes remain reusable, so verify the behavior in the serving setup you evaluate.
Explore Router