We measured time-to-first-token on 275,000 real agent requests. The benchmarks lied.

Every model release comes with a latency claim. Every leaderboard has a throughput column. And almost none of it survives contact with production traffic, because published numbers come from clean prompts, warm capacity, and whatever region the vendor benchmarked in.

We're in an unusual position to check. The Weave router sits in front of more than a dozen models and serves live coding-agent traffic through all of them, with the same instrumentation on every request. Over the last 30 days that's about 275,000 requests. Here's what latency actually looks like when real agents with real 100k-token contexts are on the other end.

The league table

Median time-to-first-token (TTFT), median total upstream latency, and blended output throughput, for every model with at least 2,000 requests in the period:

A few honest caveats before anyone quotes this table. These numbers reflect our traffic mix, not the model's intrinsic speed. Different models get routed different kinds of work, so a model that mostly handles short classifier calls will look different from one that grinds through long agent turns. The gemini-3.1-flash-lite row is the clearest example: its throughput number is tiny because we send it short-output work like titles and labels, where the response is a handful of tokens and the denominator does it no favors. Read the table as "what these models feel like in an agent harness," not as a controlled benchmark.

With that said, three findings held up everywhere we sliced them.

Fast is not one thing

The intuition that a model is either "fast" or "slow" falls apart immediately. DeepSeek v4-flash and GPT-5.5 have nearly identical TTFT, about a second, but flash streams at 131 tokens per second while GPT-5.5 does 58. If your workload is a long code generation, flash finishes in less than half the time. If your workload is a short answer where TTFT dominates, they feel the same.

That distinction matters for routing. TTFT is what an interactive user perceives. Sustained throughput is what an agent perceives, because agents read entire responses before acting. A router that treats latency as a single number will make the wrong call for one of those two audiences.

The frontier tax is real, and it's at the front

The Claude models sit at the bottom of the TTFT column, at 2.3 to 2.5 seconds to first token. Part of that is prompt-cache mechanics on huge agent contexts, and part is simply what heavy models cost in ramp-up. But notice that Sonnet 5's throughput (71 tok/s) is competitive once it gets going. The premium you pay is almost entirely at the front of the response.

Again, this shapes routing policy. For a subagent doing a quick file exploration, two extra seconds of TTFT on every turn compounds brutally across dozens of turns. For a single deep reasoning task, it's irrelevant. Same model, same latency profile, completely different verdict depending on the shape of the work.

Total latency is not TTFT plus streaming

gpt-5.6-terra starts responding in about a second but has the second-worst total latency in the table, at 7.9 seconds median, because it tends to produce long responses at moderate speed. deepseek-v4-pro is the mirror image: unremarkable TTFT, but the best total latency of the large models because its responses are efficient. If you route on TTFT alone you'd rank these two exactly backwards for agent work.

Why we publish this

Not to crown a winner. The point is that the ranking depends on the question, and the question depends on the workload. That's the whole argument for routing: no single model wins this table, so no single-model config can be right for a mixed workload. The best TTFT, the best throughput, and the best total latency in our fleet belong to three different models.

The benchmarks aren't exactly lying. They're just answering a question your production traffic never asks.