Best LLM Routers and Model Routing Tools in 2026

TL;DR

• Weave Router suits multi-harness engineering organizations that want per-action model selection tied to code quality, token costs, and delivery outcomes.

• OpenRouter Auto Router offers broad model access through one endpoint, while Cursor Router gives Cursor users native cost and quality controls.

• Ramp Router provides standalone model routing, while LiteLLM Router gives platform teams a self-hosted gateway with broad provider support.

• RouteLLM supports custom router research, Not Diamond provides managed learned routing, and Portkey adds rule-based routing to enterprise gateway controls.

• Microsoft Foundry Model Router and Amazon Bedrock Intelligent Prompt Routing fit organizations already committed to their respective cloud platforms.

Try Weave Router when you need one routing policy across coding tools and engineering outcomes.

The real cost of routing wrong

Sending every coding request to a frontier model raises token spend without guaranteeing a useful quality gain. Routine actions such as formatting code or generating boilerplate may perform similarly on a cheaper model. Complex refactors and multi-step tool calls can justify the frontier model’s higher price when its stronger reasoning prevents mistakes.

Sending every request to the cheapest model can create costs outside the model invoice. If a model edits the wrong files or returns an invalid tool call, the coding agent may retry with a larger prompt. A developer may then spend time correcting or abandoning the output. Cost per request can fall while cost per completed task rises.

A useful LLM router predicts which model can complete each request at an acceptable quality level. It sends routine work to lower-cost models and escalates requests when predicted quality separates. Provider fallback and budget rules can keep requests available or within a spending ceiling, but those controls do not establish whether the selected model can handle the work.

Router evaluations should therefore examine completed work, retries, and code quality alongside token cost. The products in this list differ most in how they make that selection and what evidence they expose afterward.

Learned routers vs. rule-based gateways

Learned selectors predict which model can meet a quality target for each request. They score prompt features or estimated difficulty before choosing among candidate models. Weave Router, OpenRouter Auto Router, Cursor Router, Ramp Router, RouteLLM, Not Diamond, Microsoft Foundry Model Router, and Amazon Bedrock Intelligent Prompt Routing belong in this group. Their scoring methods and model pools differ, so the learned label does not imply equal routing quality.

Weave Router uses an on-box cluster scorer for each upstream API request. RouteLLM applies trained preference models to a strong and weak model pair. Not Diamond supplies a learned meta-router, while OpenRouter Auto uses that technology behind its catalogue. Cursor and Ramp make quality, cost, or speed tradeoffs within their respective routing products. Microsoft Foundry and Amazon Bedrock provide learned selection inside their own cloud platforms.

Rule-based gateways execute policies that you define instead of predicting answer quality. LiteLLM Router handles load balancing, retries, fallbacks, and provider availability. Portkey evaluates request metadata and parameters against sequential conditions, then sends unmatched traffic to a default target through its conditional routing.

Both categories can support fallback and cost controls. The deciding distinction concerns the first model choice. A learned selector estimates model suitability, while a gateway follows explicit rules or provider health signals.

LLM routers and gateways compared at a glance

Routing methods range from learned quality prediction to conditional rules and provider failover. Undocumented fields remain marked rather than inferred from third-party descriptions.

What to look for in an LLM router

Routing unit. Check whether the router decides per action, per turn, or per session. Per-action routing selects a model for every upstream API request, while per-turn routing covers one user interaction. Per-session routing keeps one model throughout the working session.

Decision method. Learned selectors predict which model can handle a request at the required quality. Rule-based gateways follow conditions such as budgets or request metadata, while fallback layers primarily react to provider health.

Model pool breadth. A broad pool gives the router more price and capability options. Check support for proprietary providers and open-weight models, along with any restrictions to one cloud or model family.

State and caching awareness. Model changes can affect conversation continuity and provider prompt caches. Ask whether the router considers prior context and cache availability before switching models.

Observability. Useful telemetry should show which model handled each request and why the router chose it. Cost, latency, and failure data should remain available at the request and session levels, preferably through exports or standard telemetry protocols.

Deployment model. Hosted routers reduce operational work but place more infrastructure under vendor control. BYOK keeps provider billing under your accounts, while full self-hosting gives you greater control over routing data and credentials. Check where keys reside, how the router encrypts them, and whether its license permits your intended deployment.

Weave Router

Best for

Engineering teams that want model-routing decisions evaluated against code quality, developer activity, and delivery outcomes.

What it is

Weave Router selects a model for each action, which means each upstream API request rather than each conversation turn or session. A small on-box cluster scorer derived from Avengers-Pro predicts which model fits the request. Weave does not send the prompt through another LLM for classification.

The router acts as a drop-in proxy for Anthropic Messages, OpenAI Chat Completions, and Gemini native APIs. It supports streaming, tools, and vision. You can also reach DeepSeek, Kimi, GLM, Qwen, Llama, and Mistral through OpenRouter or another OpenAI-compatible endpoint.

You can set one routing policy per team or repository and apply it across Claude Code, Codex, opencode, and pi. Early beta support connects Cursor through a base URL override. Harness-native routers such as Cursor Router keep their policies inside one coding tool.

Pros

• The setup wizard runs through npx @workweave/router without requiring a repository clone, Docker, or Postgres. The router uses your provider keys and encrypts them at rest on your machine. A fully self-hosted deployment remains available for customers that want to operate the router and database themselves.

• Codex retains its native ChatGPT OAuth login after installation. Codex skills and Claude Code commands let you pin a model for the current session, inspect available models, or turn routing off without removing the configuration.

• Weave exposes routing telemetry through OTLP. You can inspect it in the Weave dashboard or export it to Honeycomb and other OTLP backends such as Datadog or Grafana. The /v1/route endpoint previews a routing decision without calling the selected model, while separate endpoints expose session costs, estimated savings, and raw routing decisions.

• Quality-first mode chooses a frontier model when quality scores separate and enforces a minimum quality floor. Balanced mode chooses the cheapest model inside a quality parity band, then uses the frontier model when candidates fall outside that band. Cost-first mode selects the least expensive model that clears a defined quality bar, which suits routine work at high volume.

• Weave self-reports about 3 milliseconds of routing overhead and zero application code changes for adoption. The vendor also reports 99.9 percent request completion with automatic failover. In its published routed-traffic example, Weave reports a 91 out of 100 average code-quality score and 52 percent lower cost per merged pull request than frontier-only routing. These figures come from Weave rather than independent testing.

• Weave evaluates routing against engineering data that its broader platform already collects, including code quality, pull request keep rate, developer-level attribution, and cost per merged pull request. Ramp offers a lightweight, model-agnostic router but evaluates routing as a standalone layer. Weave can therefore test whether lower token costs correspond with acceptable software delivery outcomes.

Cons

• The code uses the ELv2 license, so Weave Router is source-available rather than open source under a conventional permissive license.

• Cursor support remains in early beta and requires a base URL override.

• The strongest SDLC-based evaluation depends on using Weave’s engineering intelligence data. Buyers seeking only provider failover may prefer a narrower gateway.

• The published performance and quality figures are vendor-reported and need validation against your repositories, prompts, and model pool.

Pricing

Weave does not publish specific Router plan prices. Customers can start free or contact Weave for enterprise pricing and deployment requirements.

OpenRouter Auto Router

Best for

OpenRouter Auto Router suits teams that want a large model catalogue behind one endpoint and prefer OpenRouter to choose a model for each request.

What it is

OpenRouter exposes models from many providers through a shared API. Third-party coverage reports access to more than 400 models, though the supplied research contains no OpenRouter documentation confirming the current Auto Router pool.

A caller can request automatic model selection instead of naming a model. Available third-party examples describe cost and speed strategies that select the cheapest or fastest suitable option. OpenRouter documentation was not available to verify the current parameters, selection method, session stickiness, or distinction between openrouter/auto-beta and the reportedly deprecated openrouter/auto.

Pros

• The shared endpoint reduces integration work when you need models across many providers.

• Cost and speed strategies offer simple controls for workloads that do not require a learned quality selector.

• OpenRouter separates automatic model selection from ordered fallback chains, based on the available third-party description.

Cons

• Public details supplied for this review do not explain how Auto Router judges whether a model can handle a request.

• OpenRouter’s current model pool, routing overhead, and session behavior remain unverified here.

• Broad catalogue access does not establish that the router selects models based on code quality or delivery outcomes.

Pricing

No OpenRouter-owned pricing source was provided. A competing gateway vendor reports a 5.5 percent platform fee, but buyers should verify current fees directly with OpenRouter.

Cursor Router

Best for

Cursor Router suits teams standardized on Cursor that want model selection and cost controls across Cursor’s editor, agents, CLI, SDK, and iOS app.

What it is

Cursor Router offers three modes. Auto Cost selects a lower-cost model and charges a flat token rate. Auto Balance and Auto Intelligence can select stronger models, and Cursor bills the rate of the model used. Cursor reportedly trained the router on more than 600,000 live requests, but available sources do not document its exact decision method or routing unit.

Router policies remain inside Cursor rather than applying across separate coding tools such as Claude Code or Codex. Administrators can enable modes by group, set defaults, and block specific models.

Pros

Cursor Router gives developers model routing without adding another proxy or changing their coding interface. Cursor reported savings of 30 to 50 percent during early access and 60 percent in A/B tests, according to a third-party summary of Cursor’s rollout.

Cons

Cursor Router cannot provide one routing policy across multiple independent coding harnesses. Cursor’s public documentation supplied for this comparison also leaves the router’s selection method, latency, and routing unit unspecified.

Pricing

Auto Cost charges $1.25 per million input or cache-write tokens, $0.25 per million cache-read tokens, and $6 per million output tokens. Auto Balance and Auto Intelligence charge the selected model’s rate. Third-party model usage may also consume plan allowances or incur Cursor’s team token fee.

Ramp Router

Best for

Finance-adjacent teams that want a lightweight, model-agnostic routing layer without an engineering-intelligence product attached.

What it is

Ramp Router directs requests among AI models through a standalone routing layer. Its model-agnostic approach lets you evaluate routing separately from any single provider or coding harness. Ramp’s available product description does not specify whether the router uses a learned quality model, fixed rules, or another decision method.

Pros

• Model-agnostic routing avoids dependence on one model provider.

• A lightweight scope may suit teams that already operate their own AI gateway, observability stack, or developer tooling.

• Standalone deployment keeps routing concerns separate from broader engineering analytics.

Cons

• Ramp evaluates routing as an isolated layer. It does not connect model choices to code quality, developer-level attribution, PR outcomes, or cost per merged PR.

• The available product information does not document the routing unit, supported model pool, deployment options, or published performance metrics.

• Engineering teams may need separate tools to determine whether cheaper routing produces more retries, failed tool calls, or discarded code.

Pricing

Ramp does not provide pricing details in the available router materials. Ask Ramp for current pricing and confirm whether charges depend on request volume, token usage, or platform access.

LiteLLM Router

Best for

Platform teams that want a self-hosted gateway with broad provider compatibility.

What it is

LiteLLM provides one interface for more than 100 LLM providers through a Python SDK or an OpenAI-compatible proxy server. Its Router balances requests across deployments and retries failed calls. Configured fallbacks can send a request to another deployment or provider.

LiteLLM functions as an AI gateway rather than a learned quality selector. It routes according to configuration and provider availability, but it does not predict which model will produce the strongest answer for each request.

Pros

• The unified API covers major hosted providers and local runtimes, including OpenAI, Anthropic, Bedrock, Gemini, Ollama, and vLLM.

• Self-hosted deployment options include Docker, Kubernetes, and cloud infrastructure modules.

• Virtual keys support scoped access, while built-in cost tracking records spend by project or user.

• Guardrails and an administrative dashboard support centralized governance.

Cons

• You must define routing policy, fallbacks, and deployment priorities rather than relying on a trained quality model.

• Operating the proxy can require databases, caches, and cloud infrastructure at larger scales.

• Enterprise controls such as SSO and audit logs require the separate Enterprise Tier.

Pricing

LiteLLM publishes its core project as open source. The vendor does not publish Enterprise Tier pricing in the cited repository, so you must request a quote for commercial features and support.

RouteLLM

Best for

Research-minded teams that want an open framework for training and evaluating routers across a strong and weak model pair.

What it is

RouteLLM uses a matrix-factorization router by default to estimate whether each prompt needs the stronger model. A cost threshold controls the share of requests sent to that model. Its Performance Gap Recovered metric measures how much of the quality difference between the weak and strong models the router recovers at a given cost level.

The Apache-2.0 framework includes matrix factorization, similarity-weighted ranking, BERT, and causal LLM routers. You can run it as an OpenAI-compatible server or use it as a replacement for the OpenAI client.

Pros

• RouteLLM exposes its training and evaluation methods for modification.

• The framework provides explicit controls for studying cost and quality tradeoffs.

• The published research reports generalization to model pairs outside its original training pair.

Cons

• The included routers were trained on GPT-4 and Mixtral preference data, so you should validate performance on your own models and workloads.

• RouteLLM focuses on binary strong and weak model selection rather than broad provider orchestration.

• You must operate the service and evaluation pipeline yourself.

Pricing

RouteLLM is free under Apache-2.0. You pay for hosting and model usage.

Not Diamond

Best for

Not Diamond suits teams that want a learned meta-router across a large model catalogue without developing their own classifier.

What it is

Not Diamond trains a selector using prompts, candidate responses, and evaluation scores. The selector predicts which model should handle each request, while your existing gateway manages traffic. A third-party review reports support for more than 60 models and says Not Diamond powers OpenRouter’s Auto mode.

Pros

• The ready-made router works without customer training data, while custom routers can learn from task-specific evaluations.

• Custom training reportedly starts with about 15 labeled samples, which lowers the initial data requirement.

Cons

• Adding a new model requires retraining, and the router may remain unavailable during a training run.

• Routing decisions return a model choice without an inspectable difficulty score or similar intermediate signal.

• The available research lacks Not Diamond documentation for benchmarks, compliance certifications, and its current OpenRouter relationship. Treat third-party descriptions as unverified.

Pricing

Third-party reporting describes a free Early Access plan and custom Enterprise pricing. Not Diamond reportedly charges a fixed fee per million tokens, but no exact public rate appears in the supplied sources.

Portkey AI Gateway

Best for

Portkey suits teams that need enterprise gateway controls such as caching, guardrails, and spend management alongside predictable routing rules.

What it is

Portkey applies sequential query and then conditions rather than predicting which model will produce the best answer. Rules can inspect request metadata, model parameters, or URL paths. Gateway configurations support conditional routing, fallbacks, and load balancing, while a required default handles unmatched requests. Conditional routing works across Portkey’s catalogue of more than 250 models.

Pros

Portkey combines one provider interface with retries, caching, key management, budgets, rate limits, and guardrails. You can self-host the open-source gateway or use managed and hybrid deployment options.

Cons

Portkey does not provide a learned quality selector. You must define and maintain the conditions that send requests to each model. Portkey also reserves semantic caching, advanced access controls, and several compliance features for paid tiers.

Pricing

Portkey offers a self-hosted open-source tier without request limits. The free Dev tier includes 10,000 requests per month. Pro costs $49 per month for 100,000 requests, with each additional 100,000 requests costing $9 per month. Enterprise pricing, volume, and retention are custom.

Microsoft Foundry Model Router and Amazon Bedrock Intelligent Prompt Routing

Microsoft Foundry and Amazon Bedrock suit companies that want routing inside their existing cloud platform. Both avoid a third-party proxy, but neither offers a cloud-neutral deployment.

Microsoft Foundry Model Router

Best for: Azure users who want quality-aware routing across a model pool that includes models from several vendors.

What it is: Foundry deploys the router like a chat model and selects a model for each request. Its three routing modes favor quality, cost, or a balance of both. Custom deployments can limit the available models, while automatic failover redirects requests within that subset.

Pros: Foundry supports tool use, text and image inputs, and a broad pool that includes OpenAI models alongside Claude, DeepSeek, Llama, and Grok. Applications can call the router through the standard chat completions API.

Cons: Foundry runs only on Azure. Routing-mode changes can take up to five minutes, and Claude models require separate deployment to the Foundry resource.

Pricing: Microsoft’s supplied catalog does not state a separate router price. Check Azure pricing for the selected region and models.

Amazon Bedrock Intelligent Prompt Routing

Best for: AWS users who want serverless routing between cheaper and stronger models within one model family.

What it is: Bedrock predicts model performance for each prompt and routes within the Claude and Llama families, plus Amazon Nova. AWS offers default and custom routers. AWS reported roughly 85 milliseconds of P90 routing overhead and up to 63.6 percent savings in internal testing at general availability, according to an AWS executive’s announcement.

Pros: Bedrock keeps routing, invocation, and billing within AWS. Serverless endpoints reduce infrastructure work.

Cons: Bedrock cannot route freely across unrelated model families. Public third-party testing also reports limited visibility into decision criteria.

Pricing: The supplied sources do not confirm a separate routing fee. Model usage follows Bedrock pricing.

Which router fits your team

• Multi-harness engineering organizations should pick Weave Router. One routing policy can cover Claude Code, Codex, opencode, and other supported endpoints while connecting model choices to code quality, delivery, and cost per merged PR.

• Cursor-only shops should pick Cursor Router. Its native modes provide model selection and cost control without adding another tool outside the IDE.

• Platform teams needing broad provider reach should choose OpenRouter or LiteLLM. OpenRouter offers a managed endpoint with a large model catalogue, while LiteLLM suits teams that want a self-hosted gateway and control over provider configuration.

• Azure-committed teams should use Microsoft Foundry Model Router, and AWS-committed teams should use Amazon Bedrock Intelligent Prompt Routing. Both keep routing inside the existing cloud platform, but each limits model choice to its supported ecosystem.

• Teams that want to train and evaluate their own selector should pick RouteLLM. Its open framework supports custom strong-versus-weak model routing experiments rather than prescribing a managed routing policy.

Why Weave Router leads

Weave Router leads for engineering organizations because it checks model selection against the work developers produce. Weave connects routing decisions to code quality and PR keep rate. It also tracks cost per merged PR with developer-level attribution, so you can see whether lower token spend translates into efficient software delivery.

The Balanced policy illustrates the approach. Its scorer chooses the cheapest model inside a quality parity band and sends requests to a frontier model when quality scores separate. You can then inspect routing decisions alongside downstream results instead of treating token cost or latency as sufficient evidence.

A gateway may fit better when you need provider failover or broad API compatibility alone. Weave Router earns the top pick when you need one routing policy across coding tools and want to verify its effect on engineering performance.

Try Weave Router with your existing coding setup.

FAQs

What is an LLM router?

An LLM router selects a model for each request based on factors such as predicted quality, cost, or availability. Weave Router scores every upstream API request with an on-box cluster scorer. Request-level selection can reduce model spend while preserving stronger models for harder work.

How does an LLM router differ from an AI gateway?

An LLM router chooses which model should handle a request, while an AI gateway manages access, retries, limits, and provider failures. Weave Router combines quality-based selection with proxy and automatic failover capabilities. You should check whether a product predicts model quality or merely applies fixed rules.

Is Weave Router self-hosted or open source?

Weave Router supports hosted and fully self-hosted deployment, and its visible code uses the source-available ELv2 license rather than an open-source license. Weave also uses BYOK by default and encrypts provider keys on your box. You can inspect the code and choose the deployment model that fits your security requirements.

Does model routing hurt code quality?

Model routing can reduce quality when a router sends difficult requests to models that cannot handle them. Weave’s Balanced policy selects the cheapest model within a quality parity band and uses a stronger model outside that band, while its other policies enforce quality floors. You can control savings without treating every request as equally difficult.

Can you force a specific model when the router picks wrong?

Model pinning overrides automated selection for a defined session. Weave Router supports per-session model forcing and lets you turn routing off without removing its configuration. You can keep automated routing as the default while retaining manual control for sensitive work.