Measuring AI Coding ROI in Shipped Work
Measuring AI Coding ROI in Shipped Work
AI usage is easy to count. AI ROI is harder because the spend and the resulting work usually live in different systems. A credible measurement joins an explicit cost source to a defined output cohort, then places quality and delivery context beside the ratio.
The short answer
Weave's Token Intelligence and Engineering Intelligence products connect supported AI usage and cost data to pull requests, reviews, deployments and output measures. The result is an investigation surface for AI spend and shipped work. It is not a claim that every prompt caused a specific change or that an output estimate equals cash savings.
Define the unit before collecting data
Choose one decision, such as renewing a coding tool, changing model use or comparing foreground assistants with background agents. Record the repository set, team scope, time window and exclusion rules.
Then define the output unit. Weave's public methodology uses estimated expert engineering hours for its cost-per-output-hour series. A merged PR, an expert-hour estimate, a deployment and a billed dollar are different units. Keep them separate in the report.
Reconcile the cost sources
Token Intelligence can bring together provider billing, tool telemetry and configured subscription costs for supported integrations. Cost sources need labels.
- Billed actuals come from provider or tool billing data.
- Configured subscription costs represent a seat or plan allocation.
- Modeled or estimated values use a documented method when direct billing is unavailable.
- Organization-level charges may not be assignable to one engineer.
Do not add a subscription seat cost to a provider charge when the included usage is already represented. Keep unassigned spend visible beside the matched cohort instead of silently distributing it.
Match spend with shipped work
Use the same engineers, repositories and period for cost and output. Map identities across the coding tool, source control and engineering systems. Mark the evidence behind each AI contribution as native integration telemetry, commit metadata or an activity-based estimate.
Inspect representative pull requests. A routine generated change and a multi-file migration both count as one PR, but they do not represent the same amount of work. Weave's code-output measure estimates expert effort from the merged change and exposes the reasoning so the team can review the unit.
Calculate the ratio for a matched cohort
Suppose a team has $1,200 in attributed AI spend and 300 estimated expert hours of merged output for the same month. That is $4 per estimated output hour, or 0.25 estimated output hours per dollar. This example is hypothetical. An estimate of 300 expert hours describes the work represented by those merged changes. It does not measure how many hours anyone saved.
Add quality and delivery context
A lower cost ratio can accompany more rework. Review code quality, review cycles, reverts, production defects and deployment measures alongside spend and output. Recent changes may not have enough time to show their full post-merge outcome.
DORA and SPACE measures provide delivery context. They do not prove that AI caused a change. Record staffing, project scope, release cadence and other changes that could affect the comparison.
A practical evaluation sequence
- Choose the renewal or rollout decision.
- Freeze the cohort, repositories and period.
- Reconcile provider charges, subscriptions and estimates.
- Check identity and integration coverage.
- Calculate cost and output from matched totals.
- Inspect representative PRs and quality signals.
- Compare a consistent baseline or comparable group.
- Record exclusions, unknowns and assumptions beside the result.
For a financial ROI percentage, finance must define incremental benefit and cost. An estimated expert-hour output is not recorded time, payroll savings or proof that AI caused the result.
Frequently asked questions
Does cost per merged PR equal ROI?
No. It is a cost ratio with a defined denominator. Use it with the PR population, output unit and quality context that explain what the ratio means.
Can every token be assigned to a person?
No. Coverage depends on the tool integration and billing source. Some tools report organization-level spend, and shared or unconnected work can remain unattributed.
Can Weave show what an AI-assisted change produced?
For connected records, Weave can place AI usage, cost, code output, review and delivery context together. Check the source labels and inspect the underlying records before making a decision.
Where are the definitions?
Use the AI ROI methodology, Token Intelligence and Engineering Intelligence pages as the current references.
Conclusion
AI ROI becomes useful when a team can explain the denominator, trace the evidence and see the work behind the number. Start with one decision and one matched cohort. Then improve coverage before expanding the claim.