Agent Router

What is being routed?

The term can describe two different decisions. Task routing chooses a worker or workflow, such as a documentation agent versus a code-repair agent. Model routing chooses the model that handles a particular inference request. A system may perform both, but choosing a model does not automatically delegate the entire task.

Anthropic describes routing as classifying an input and directing it to a specialized follow-up. This is useful when distinct request categories benefit from different handling. Anthropic's routing pattern.

Document the unit of decision. Is the destination fixed for a task, a conversation, or one turn? Changing destinations mid-task can affect available context, supported tools, and how prior reasoning is represented. A provider's routing documentation should explain those behaviors rather than leave users to infer them from a single endpoint name.

A hypothetical coding workflow

Imagine a repository assistant that receives three requests: update a changelog, fix a race condition, and explain a deployment failure. A task router sends the first to a documentation workflow, the second to a coding workflow with test access, and the third to an investigation workflow with read-only logs.

Within the coding workflow, a model router might use one model for a bounded code search and another for a difficult repair. The harness still decides which tools are available and whether the proposed edit may execute. Routing to a more capable model does not grant additional permissions.

A routing record should retain the selected destination, policy version, and outcome. Without that connection, a reduction in model spend cannot be associated with the tasks that succeeded or needed human repair.

Policies and constraints

Some routers apply fixed rules; others classify requests with a learned model or compare expected cost and quality. Before optimization, filter out destinations that cannot satisfy hard requirements such as tool compatibility or an organization's data-handling restrictions.

Check whether a product chooses models or providers, and how it behaves when a chosen destination fails. OpenRouter's Auto Router documentation is an example of a public specification that describes automatic model selection and its settings. The existence of an automatic route does not establish that it is best for every coding workload. OpenRouter Auto Router documentation.

Measure outcomes, not just request prices

Compare total task cost, accepted results, latency, retries, and review effort against a fixed-policy baseline. A cheaper first call can increase overall cost if it requires repeated repairs. Include the router's own overhead and unsuccessful work.

Use representative tasks and preserve the same harness and acceptance rules across candidates. Report results by task type so a gain on documentation does not conceal a regression on complex changes. Agent run comparison provides the experimental structure; model routing describes the narrower inference-selection mechanism.