← All buyer guides

AI Cost Observability Tools: Coding Agents, LLM Apps and Cloud Spend Compared

Compare AI cost observability tools by what they attribute: coding-agent spend in Weave, LLM requests in gateways and tracers, cloud and GPU bills in FinOps tools.

By Co-founder CEO
Published Updated Read Time: 7 min
On this page

AI cost observability tools fall into three groups that answer different questions. Request-level tools and gateways such as Langfuse, Helicone, Portkey and LiteLLM record the tokens and cost of each LLM call your applications make. Cloud FinOps platforms such as nOps, CloudZero and Vantage allocate cloud, GPU and provider bills to teams and products. Weave covers the AI coding tools your engineers use and connects that spending to the code it helped produce.

Weave reports AI cost per person, tool and model beside AI-attributed engineering output, labels each cost by where it came from, and offers Weave Router to change the model mix when spending is not justified. Weave wrote this guide. It explains when each category fits and what to audit for coding spend specifically.

Why AI coding spend is hard to read

Coding spend arrives in several forms on different invoices. Seat subscriptions for Cursor, Copilot or Claude plans are flat. Usage beyond a plan becomes overage. Coding CLIs may use subscription capacity or metered API access, while cloud agents may bill in credits or compute units. Prompt caching changes token prices within a session, and agents that retry or loop consume tokens without producing a merged change. Provider dashboards can expose usage and spend by user, key or project. Weave adds the engineering context: how that spend relates to code output, review and the work that merged.

Token counts multiplied by list price can overstate cost for subscription users. An engineer on a flat plan can generate list-price usage well above what the company pays. Treating that figure as spend inflates cost per output and can make heavy subscription users look expensive when they are among the best value on the team.

How Weave attributes coding costs

Weave labels each cost figure with its fidelity: actual vendor billing, estimated tokens at list price, plugin-observed plan or overage status, plugin-estimated usage, client-reported cost, off-workspace activity, modeled credits, or activity-only rows with no cost. Seat subscriptions appear separately in the per-person drilldown; cost-per-output totals include prorated seat costs. Finance can reconcile the report with invoices using those labels before it informs a budget decision.

Coverage depends on the tool. Current cost documentation includes Claude Code and Claude.ai, Cursor, GitHub Copilot, Codex, the OpenAI API, OpenRouter, Devin, Windsurf, Gemini CLI, Amazon Q, Sourcegraph, Augment, Kiro, Cline, Firebender, Oh My Pi and Modal. Modal reports at organization level only. Gemini CLI and Cline need configured subscriptions before Weave can show dollar cost, and OpenRouter per-person spend requires identity mapping. Check the AI tool cost documentation for every tool on your contract list.

On the output side, Weave attributes AI contribution using the strongest available evidence: direct tool API data, then line-level git-ai notes, then co-author metadata, then activity inference. The engineer who drove AI-assisted work still receives output credit. AI attribution describes how code was produced, not who gets credit for it.

Compare AI spend with engineering output

Weave's AI report relates attributed AI spend to AI-attributed output measured in estimated expert-engineer hours, with drilldowns by person, tool and model. The public Weave Index reports a separate cohort statistic: estimated expert output per attributed AI dollar. Its October 7 snapshot reports 0.090777 estimated expert hours per attributed dollar for August 2026, calculated for a matched cohort of engineers with both spend and output in that month. The figure is modeled output per dollar. It is not a financial ROI percentage or a count of hours saved, and your own ratio is more useful as a trend than as a comparison with the Index.

Intellistack connected GitHub, Cursor and Claude Code to Weave to analyze its AI adoption historically and report it to its board.

Reduce AI coding spend with Weave Router

Cost reporting identifies the expensive sessions and tools. Weave Router changes the cost of the next request by selecting a model for each coding request based on task difficulty, cache-switch cost, progress and subscription capacity. Weave's Router 2.0 announcement reported 40% lower token spending on average across its production aggregate. Router is billed separately from Weave's analytics plans, and its reporting estimates savings against a fixed comparison model you choose. See Weave Router for plans and setup.

Which category fits which spend

Scroll horizontally if needed →

CategoryExamplesWhat it attributesChoose it when
Coding cost and outputWeaveAI coding tool spend by person, tool and model, with source labels and attributed code output in expert-effort unitsEngineering leaders need to justify, reallocate or reduce AI coding budgets
Engineering analytics with AI cost viewsJellyfish, DX, LinearB, SwarmiaAI cost beside throughput, merged PRs or stories, using each vendor's own cost methodYou already run one for delivery reporting and want cost added to it
Request-level tracing and gatewaysLangfuse, Helicone, Portkey, LiteLLMTokens and cost per request, tagged by user, feature or customerYou build LLM features and need per-request cost, budgets and limits
Cloud and AI FinOpsnOps, CloudZero, VantageCloud, GPU and provider bills allocated to business units and productsFinance owns total cloud and AI infrastructure spend

The engineering analytics vendors differ in their denominators. DX reports cost per merged PR, LinearB divides people-plus-AI cost by rework-adjusted PRs, Swarmia uses configured labor cost and token list prices, and Jellyfish compares token spend with output. Weave focuses on engineering and AI-tool economics and also ingests organization-level compute billing through integrations such as Modal. Teams needing broader cloud and GPU allocation may use a FinOps platform alongside it. Teams building LLM products keep a request-level tool for that traffic.

Compare your AI spend and output

This scenario compares AI spend with all measured merged output from the same people and period. Weave's in-product AI report uses AI-attributed output instead, so these ratios answer a different question. Include applicable seat costs once and label estimates separately from billed amounts.

  • AI cost per estimated total-output hour = attributed AI spend ÷ estimated expert hours of all merged output.
  • Estimated total output per dollar = estimated expert hours of all merged output ÷ attributed AI spend.

These ratios compare cost efficiency. They do not establish cash savings or isolate the effect of a tool. The calculator below calculates these ratios and a separate routing-cost comparison.

Compare attributed AI spending with estimated engineering output, then explore a separate routing-cost scenario. Keep the people, repositories and month consistent across each calculation.

This calculator uses all measured merged output for the selected people and period. The in-product AI report divides AI costs by AI-attributed code output, so the two ratios differ.

Illustrative inputs. These are fictional examples, not customer results or benchmark targets.

Your measured group

Attribute costs to people who also have measured merged output in the same period.

Include attributed usage and applicable seat costs. Avoid counting included usage twice.
Estimated effort represented by all merged work, not recorded time or hours saved.
Use a consistent token-count basis, including cache categories consistently.

Three distinct ratios

AI cost per estimated total-output hour$4.00Attributed spend ÷ estimated expert hours of all merged output
Estimated total output per dollar0.2500Estimated expert hours of all merged output ÷ attributed spend
Estimated total output per million tokens50.00Estimated expert hours of all merged output ÷ (tokens ÷ 1,000,000)

These ratios are not financial ROI, hours saved, or proof that AI caused the output. Review quality, rework and delivery alongside them. Output per million tokens is a scenario calculation here; it is distinct from the Weave Index’s output-per-dollar metric.

Compare a routing-cost scenario

Use the same task set, acceptance criteria, attempts and cost basis for both amounts. This comparison does not assume equal coding quality or change the output inputs above.

Cost for the fixed comparison task set.
Cost for the same task set using routing.
Baseline$1,000.00
Routed$750.00

$250.00 lower spend. 25.0% reduction from baseline.

Difference = baseline − routed. Percentage = difference ÷ baseline × 100.

What Weave observed in Q3 2026

Weave’s Q3 2026 report covers 22,623 engineers. Its platform-wide monthly averages show output per employee rising from 70 to 131 units, while average AI spend rose from $231 to $575.

Published platform-wide averages, separate from the illustrative inputs above.
MeasureJune 2026September 2026
Average monthly output per employee70 output units131 output units
Average monthly AI spend per employee$231$575

The report’s blended monthly averages and the Index’s matched spend-and-output cohort use different populations. Dividing these displayed averages does not reproduce the Index’s cost-per-output ratio. This observational trend does not show the causal effect of buying Weave.

The calculator does not submit these values. Reconcile billing, check telemetry coverage and inspect the underlying work before using a result.

A coding cost audit before renewal

  1. List every AI coding contract: seat plans, overage terms, API keys and cloud agent credits.
  2. For each tool, record whether cost is billed, estimated or subscription-based, and whether it is reported per person or per organization.
  3. Map identities across source control, coding tools and API keys so spend and output join on the same person.
  4. Choose one month and compare AI cost per output by tool and team. Include prorated seat costs in the total, and show seat and usage amounts separately.
  5. Open the ten most expensive sessions or users and inspect the PRs they produced, their review rounds and any reverts.
  6. Decide per tool whether to renew, reallocate seats or route requests to cheaper models.

Connect source control and two or three coding tools, reconcile one month against your invoices, and compare cost per output by tool. See Token Intelligence or book a demo.

See the work behind your AI spend

Compare code output, AI cost and quality, then inspect the work behind the trend.

Book a demo