Provider routing
Also known as LLM provider routing, Multi-provider routing
Definition
Provider routing selects an infrastructure provider or endpoint for a model request. The policy may consider price, latency, availability, regional requirements, quotas, capabilities, or recent provider performance.
Provider and model are separate choices
The same model family may be available from several providers. Each endpoint can have different pricing, rate limits, regions, context support, and serving behavior. Provider routing chooses where the request runs after the eligible options are known.
A model router may choose the model first and then let a provider policy choose the endpoint. Some systems make both choices together. Name the order in documentation so operators can understand which policy produced a request.
Signals need a time window
A provider's current latency or error rate can inform a routing rule, but a single slow request is weak evidence. Track rolling rates and latency percentiles with enough request volume to avoid reacting to noise. Record policy versions and decision reasons for later analysis.
Keep capability checks ahead of savings
The cheapest endpoint is not suitable if it lacks a required region, tool interface, context length, or output format. Validate capabilities before comparing price. Then compare complete task cost, success, retries, and time to useful completion across the available providers.
Fallbacks can help during an outage, but they should be measured separately from normal routing. A high fallback rate can signal a capacity or configuration problem that deserves investigation.
How this relates to Weave
Weave Router focuses on selecting models for coding-agent work. Provider routing can be a separate decision or a layer beneath model selection. Evaluate the combined path because provider behavior affects the latency, price, and reliability users experience.
Explore Router