Time to first token
Also known as TTFT, First-token latency
Definition
Time to first token, or TTFT, is the elapsed time from an inference request being accepted until the first output token is delivered. It captures startup and queue delay before generation becomes visible to a user.
What TTFT includes
TTFT includes request admission, queueing, input processing, and the time needed to begin decoding. It does not describe how quickly the remaining response streams. A response can have a fast first token and a slow completion, or the reverse.
Why users notice it
Streaming interfaces feel responsive when useful output begins quickly. TTFT is especially visible in chat, coding assistants, and agent steps where one delayed call delays the next tool action. Set targets by use case rather than applying one number to every request.
Improve and interpret it
Reduce unnecessary context, use prompt caching where appropriate, route urgent requests to providers with available capacity, and monitor queue time separately. Weave connects TTFT to tokens, model routes, and outcomes so teams can optimize perceived speed without ignoring quality or cost.
How this relates to Weave
Weave helps teams compare TTFT by model, provider, route, prompt size, and cache state. That context can show whether a slow first token comes from queueing, a long input, a provider, or a routing rule.
Explore Token intelligence