Model fallback
Also known as LLM fallback, Provider fallback
Definition
Model fallback is the use of an alternate model or provider when the preferred route cannot handle a request or fails a defined condition. A fallback policy can respond to outages, rate limits, unsupported capabilities, timeouts, or application-level checks.
Fallback is a recovery policy
Routing chooses a candidate before or during a request. Fallback takes effect when a condition says the candidate cannot complete the work. A provider timeout, a missing tool-calling capability, or a rejected context length can each trigger a different response.
The condition should be explicit. Retrying every failure against the same model may waste time. Switching models after a content refusal may be appropriate in one application and unsafe in another.
Avoid hiding the extra work
Suppose an illustrative request first goes to a low-cost model, times out, and then succeeds with a second provider. The completed task includes both attempts in its cost and latency. Reporting only the successful call makes the fallback policy look better than the user's experience.
Record the initial route, reason for fallback, alternate route, retries, and final outcome. Keep provider error categories stable enough to analyze.
Test the policy under realistic failure
Inject rate limits, slow responses, and unsupported requests in a controlled evaluation. Check that the fallback preserves required capabilities and does not create duplicate side effects. Compare cost per completed task, not just first-call price.
How this relates to Weave
Weave Router can be evaluated not only on its first model choice, but on what happens when that choice is unavailable or insufficient. Include fallback calls, their costs, and their effect on task completion when measuring a routed workflow.
Explore Router