Model gateway
Also known as LLM gateway, AI gateway, Model API gateway
Definition
A model gateway is a service layer that gives applications a common interface to one or more model providers. Depending on its design, it can handle routing, authentication, retries, fallbacks, usage tracking, and request policies.
The gateway is an application boundary
Without a gateway, application code often carries provider-specific endpoints, credentials, request formats, retry rules, and usage accounting. A gateway can place those concerns behind one interface so the application can change providers with less code.
That boundary is useful for policy too. Teams may enforce allowed models, redact selected fields, attach request metadata, or record the selected provider. The gateway becomes a place to make those rules visible and consistent.
Gateway and router are related
A gateway may route requests, but the terms describe different responsibilities. Gateway refers to the serving and policy layer. Routing refers to selecting a model or provider based on a rule or signal. A gateway can pass every request to one model, while a router can make choices through a gateway.
Observe the complete request
Record the model, provider, route decision, retries, fallback reason, input and output usage, latency, and task outcome. This makes it possible to explain a cost increase or a quality change after a configuration update.
Standardize the interface carefully. Provider differences in tool calling, context limits, streaming, and structured output can still affect application behavior behind a common endpoint.
How this relates to Weave
Weave Router is designed for model selection inside coding-agent workflows. A gateway can provide the surrounding provider interface and operational controls, while a router supplies the policy that chooses a model for a request or task step.
Explore Router