How to Control Enterprise AI Costs With Model Routing and Token Tracking

TL;DR

• Enterprise generative AI spending reached $37 billion in 2025, up 3.2 times year over year.

• Model prices can vary 10 to 30 times across comparable capability tiers, so you need routing and cost monitoring to control spend.

• You should apply static routing and prompt caching first. These measures typically capture more of the initial savings and are easier to evaluate than a learned router.

• A learned router can capture incremental savings by matching request complexity to quality per token. Per-team and per-project cost attribution then shows where inefficient usage remains.

Why AI infrastructure spend is outgrowing every budget model

Enterprise generative AI spending reached $37 billion in 2025, up 3.2 times over 2024 and more than 20 times over 2023, according to Menlo Ventures. Infrastructure accounted for $18 billion, including $12.5 billion spent on foundation model APIs. Departmental spending reached $7.3 billion, and coding tools captured 55 percent of that total. Vertical AI spending reached another $3.5 billion, with healthcare accounting for about $1.5 billion.

Traditional cloud budgets estimate usage through provisioned compute, storage, and request volume. LLM APIs charge for the tokens each request sends and generates, so two requests to the same endpoint can incur very different costs. Model choice adds another variable because token prices can vary 10 to 30 times across comparable capability tiers. According to published token benchmarks, output tokens typically cost four to five times more than input tokens across providers.

Agentic applications make request counts even less useful for forecasting. A single user action may trigger repeated model calls while an agent retrieves context and uses tools. Retries and expanding conversation histories add more tokens. Agentic tasks can consume far more tokens than standard chat interactions because one user action may trigger retrieval, tool use, retries, and repeated model calls.

Raw token estimates also omit system prompts, retries, and context overhead. A practical budget should include an explicit allowance for system prompts, retries, and context overhead based on measurements from representative workloads. Engineering leaders therefore need to forecast spend by task shape, model, and token direction rather than multiplying monthly request volume by one average price.

Model routing: matching task complexity to model cost

Model routing should begin with fixed task tiers. You can send extraction, classification, and simple support questions to a lower-cost model while reserving a frontier model for ambiguous requests, complex code, or high-risk decisions. Static rules are easy to inspect, test, and override. They also establish a cost and quality baseline before routing logic becomes harder to interpret.

Several cost controls should precede dynamic routing. Prompt caching avoids repeatedly processing shared instructions or context, while reasoning-effort caps limit excess computation on routine requests. Batch APIs reduce prices for work that does not require an immediate response. One analysis of production routing strategies attributes 50 to 70 percent of available savings to these controls and static tiering rather than learned routers.

Dynamic routing adds value when fixed categories cannot capture variation within a task. A complexity classifier can inspect each request and select a model tier. A cascade can try a cheaper model first, evaluate its response, and escalate uncertain cases. A learned router can use evaluation results and production feedback to adjust its choices over time. Each step requires stronger evaluation data because routing errors become less visible as the policy grows more adaptive.

Required quality thresholds should determine how aggressively you route requests to cheaper models. A cheap model that returns an incorrect answer can trigger retries, human review, customer support work, or a bad downstream decision. Those costs can exceed the savings from avoiding one frontier call. You should therefore send uncertain requests to the stronger model until production evaluations show that a cheaper option meets the required quality threshold.

Price data alone cannot support that decision. A quality-per-token measure compares model cost with task-specific success, such as extraction accuracy, accepted code changes, or support-resolution rates. You can calculate the measure for each task class, then route requests toward the least expensive model that consistently clears its quality threshold. Regular sampling and regression tests should check whether model updates, prompt changes, or new request patterns have weakened that threshold.

Tracking token cost by developer, team, and project

Provider invoices show total spend, but they rarely identify the code path, product, or owner that generated it. Useful cost attribution starts at the request level. Each request should record the developer or service owner, team, project, environment, model, input and output tokens, cache status, retries, and latency.

Request-level data lets you enforce routing policies as budgets rather than suggestions. For example, you can see whether a support workflow exceeds its budget because it sends routine classifications to a frontier model. You can also separate deliberate exceptions from accidental ones, such as a staging service using a production API key or an agent retrying failed calls without a limit.

Token counts alone can conceal the source of rising costs. Output tokens typically cost four to five times more than input tokens, and retries or long context windows can multiply the bill without increasing completed work. Cost reports should therefore separate input, output, cached, and retried tokens. They should also compare spend with a useful unit such as resolved tickets, merged pull requests, completed evaluations, or successful agent runs.

Shared API keys make attribution harder because the provider sees several workloads as one customer. You can preserve ownership by adding internal request metadata at the gateway or SDK layer and mapping service accounts back to teams and projects. Consistent tags make fine-grained dashboards useful because missing ownership data leaves spend unallocated.

You can build this observability layer with gateway logs, provider pricing tables, metadata standards, and a data warehouse. You must then maintain pricing changes, model aliases, retries, and organizational mappings. Weave offers Token Intelligence as an alternative for tracking usage, cost, efficiency, and quality across developers, teams, projects, tools, and providers.

Granular reporting also improves routing decisions. A cheap model may lower token cost while increasing retries or producing work that developers discard. Pairing cost with completion and quality signals helps you measure cost per useful outcome, which gives a quality-aware router better evidence than provider prices alone.

Benchmarking AI spend against peer companies

Public spending data provides directional benchmarks, but it cannot define a precise target for your company. Enterprise generative AI spending reached $37 billion in 2025, 3.2 times the 2024 total. A budget growing at a similar rate may reflect broad adoption rather than poor cost control. You still need usage and outcome data to determine whether the added spend produces useful work.

Department shares can reveal where your allocation differs from the wider market. Coding accounted for 55 percent of departmental AI spending in 2025, while IT received 10 percent and marketing received 9 percent. Compare engineering's internal share with the market figure, then identify whether code completion, review, or agent workflows account for the concentration. If another department dominates, verify that its workload and measured benefits justify the difference.

Industry mix also affects a useful comparison. Vertical AI spending reached $3.5 billion in 2025, and healthcare captured about 43 percent of that amount. A healthcare company with expensive clinical workflows should not compare its AI budget directly with a software company using coding assistants. Compare against companies with similar workflows, regulatory constraints, and model requirements.

The cited market data does not provide benchmarks for AI spend per developer, seat, or successful task. Build internal benchmarks while the external data matures. Track cost per active developer and successful task, then compare those figures by project over time. Weave can connect token spending with engineering output and organizational benchmarks, but those comparisons should not be treated as market-wide token cost standards.

Comparing cost-tracking and routing platforms

Choose a platform according to the cost decision you need to make. These vendors solve overlapping problems, but they collect different data and operate at different points in the stack. Finout, CloudZero, and Vantage are cloud FinOps tools extending into AI spend, Amnic and Helicone are LLM-focused dashboards and gateways, and TrueFoundry is an enterprise AI gateway. Weave is the outlier in this set: an engineering intelligence platform with model routing built in, not a FinOps tool competing for the same budget line as CloudZero or Finout. Feature counts can obscure that distinction because a billing dashboard and a request router may both report token cost while serving different operational needs.

Platform breakdown: best fit by use case

Finout fits the broad cloud FinOps use case, where finance and engineering need one view across infrastructure and AI spending. Its value lies in multi-source cost management rather than request-level model decisions. When evaluating Finout, verify whether its allocation data reaches the model, prompt, and token level and whether it supports routing decisions.

CloudZero suits companies that need to assign shared infrastructure costs to products, customers, or engineering groups. Its Dimensions model connects billing data with business activity, which supports unit metrics such as cost per customer or feature. CloudZero has added native AI cost sources and real-time AI spend collection, according to a detailed product review. Evaluate CloudZero primarily for multi-source cost allocation, and verify request-trace depth or quality-based routing separately if you need those capabilities.

Vantage works best when you need billing-based visibility across cloud services, SaaS products, and direct AI providers. Its AWS Marketplace listing documents cost reporting, allocation, budgets, and unit-cost analysis. Vantage has also announced LLM Token Allocation through its own product coverage. An independent comparison notes that billing-centric reporting cannot diagnose why a particular request or agent workflow became expensive.

Amnic belongs on a shortlist for buyers seeking an LLM cost dashboard rather than a wider cloud FinOps suite. A dashboard-first tool can provide a simpler path to provider and model spending reports when routing stays inside the application. When evaluating Amnic, verify its allocation fields, alerting, routing support, and pricing directly.

Helicone suits application teams that need request-level LLM observability with gateway controls. Its searchable logs connect prompts and responses with token cost, latency, and custom metadata, while its gateway supports caching, rate limits, and fallbacks. Helicone also offers an open-source deployment path, according to an independent review. If you evaluate Helicone, measure the proxy's latency in your environment and account for the storage and maintenance required by self-hosting. Helicone does not replace company-wide cloud allocation.

TrueFoundry suits enterprises that need governed model access and flexible deployment. Its gateway supports policy rules, quotas, guardrails, and customer-controlled environments, including air-gapped infrastructure. An independent comparison describes its routing as rule-based around factors such as latency, load, and geography. Buyers seeking routing based on measured answer quality would need another evaluation layer.

Weave sits in a different category than the cloud FinOps tools above. Finout, CloudZero, and Vantage allocate broad infrastructure and SaaS spend across the business, and Weave does not compete for that budget line. The closer comparison is a model-agnostic router like Ramp's, which picks a model per request but stops there.

Weave installs as a one-line addition to an existing Claude Code or Codex setup and ties routing to the rest of the software delivery lifecycle instead of treating it as a standalone bill-reduction tool. Prompt Router uses quality-per-token data rather than price alone to select models. Token Intelligence then tracks usage, cost, and efficiency so you can connect routing decisions to developer output, code quality, and delivery data, by developer, project, and engineering group. Teams that need broad cloud FinOps should still use Finout or CloudZero. An LLM-only dashboard such as Amnic may suit buyers who need spend reporting without an engineering-intelligence layer underneath it.

Comparison table

Putting a routing and monitoring program in place

Treat routing and cost monitoring as one operating loop. Begin by tagging each request with its owner, project, use case, model, token volume, cost, latency, and quality outcome. Use that baseline to identify repeatable workloads that meet their quality threshold on lower-cost models and workloads that still require frontier models.

Apply static routing, prompt caching, reasoning caps, and batch APIs before adding learned routing. Compare every policy change against cost and quality baselines. A policy that lowers token spend while increasing retries or rework fails the economic test. Bias thresholds toward quality until production evaluations show that a cheaper model meets the required standard.

Combine routing with cost attribution so you can lower request prices, assign spend to accountable budgets, and correct inefficient model use. Review routing thresholds, evaluation sets, and budgets whenever usage patterns change or new models become available.

Weave combines quality-per-token decisions through Prompt Router with usage, cost, and efficiency tracking through Token Intelligence, installed as a one-line addition to Claude Code or Codex rather than a separate cost dashboard to maintain. Teams that want routing tied to engineering output, not just a lower model bill, can start a free Weave account or book a demo.

FAQs

How much can model routing actually save?

Model-routing savings depend on workload mix, model price differences, and quality requirements. Research attributes 50 to 70 percent savings mainly to static tiering, caching, reasoning limits, and batch processing before learned routing. Weave can measure quality per token so you can identify incremental savings without sending unsuitable tasks to cheaper models.

How does model routing differ from prompt caching?

Model routing selects a model for each request, while prompt caching reuses previously processed input. Weave’s Prompt Router can complement provider caching by choosing a suitable model after cached input savings apply. Using both controls reduces repeated input charges and unnecessary frontier-model calls.

How do you measure AI ROI per developer or team?

AI ROI compares attributable AI spend with changes in engineering output and quality. Weave’s Token Intelligence connects developer, team, and project costs with delivery and code-quality data. You can identify where higher token consumption supports measurable gains and where it produces little operational value.

How often should routing rules be reevaluated?

Routing-rule reevaluation tests whether current model assignments still meet cost and quality thresholds. Weave can track quality-per-token changes as providers release models or revise pricing. Review routing rules monthly and test them again whenever a provider changes a model or its pricing. These checks help you update policies before outdated assumptions affect production traffic.