Model routing
Also known as LLM routing, AI model routing, Intelligent model routing
Definition
Model routing is the process of selecting which AI model handles a request or a step in a workflow. A routing policy can consider the task, required capabilities, expected quality, price, latency, and provider availability.
Why one workflow can need several models
A coding agent rarely performs just one kind of task. It might read a file, explain a test failure, choose an implementation approach, edit code, and summarize what changed. Those steps differ in difficulty and in the consequences of a mistake.
Using the same model for every step is simple. Routing introduces another option: choose a model that fits the current request. Routine transformations might not need the same capabilities as an ambiguous design decision involving several files.
The benefit depends on whether those differences are predictable enough to act on. If the router regularly sends difficult work to a model that cannot complete it, retries can erase the expected savings.
How the decision is made
A routing policy may use explicit rules, such as selecting a model that supports images when an image is attached. It may classify the request, estimate difficulty, or use results from past evaluations. Some systems make the decision once per conversation. Others reconsider at each turn.
Capability checks should happen before optimization. A cheap model is not a valid candidate if it lacks a required tool-calling interface or cannot accept the request's context. Provider restrictions and application requirements can further narrow the candidates.
The remaining choice is a tradeoff. Lower expected token cost may come with more output, longer response times, or a greater chance of needing another attempt. Evaluate the full workflow rather than the advertised price of one call.
A coding-agent example
Consider an illustrative agent fixing a failing test. First it lists the relevant files. Then it inspects an error trace, finds a subtle state-management bug, proposes a correction, and runs the test again.
A routing policy could use one model for the routine file-summary step and another for reasoning about the state transition. The important result is whether the final patch works, not whether every individual call was inexpensive.
Compare the routed workflow with a fixed-model baseline on the same tasks and success criteria. Include unsuccessful attempts, routing overhead, and tool execution in the measurements. A policy that looks good on isolated prompts may behave differently during a long agent session.
Routing, fallback, and load balancing
These mechanisms are related but serve different purposes. Routing selects a model for a request. Fallback chooses an alternative after a failure or another triggering condition. Load balancing distributes work across serving capacity, which may involve multiple endpoints for the same model.
A gateway can provide all three. Their presence does not prove that it chooses models based on task difficulty. Ask which decision is actually being made and what information drives it.
Keep evaluating after deployment
Task mix changes. Models and provider behavior change too. Record the chosen model, the policy version, costs, latency, and whatever outcome signals your application can obtain. Those records help explain whether a routing change improves task completion or merely shifts spending between providers.
How this relates to Weave
Weave Router applies model selection to coding-agent workflows. It examines turns within a task and chooses models with the aim of balancing cost, speed, and the ability to complete the work. Its published benchmark results provide a starting point for evaluation, while your own tasks establish whether the same tradeoffs hold in your environment.
Explore Router