How to show that AI is working

They ask whether the spend buys more of what the business needs, whether it costs reliability, and what the leader will change next. Few of them ask how many tokens the team used.

The question is fair, because most AI spending has not paid back yet. IBM surveyed 2,000 CEOs in 2025. They reported that 25% of their AI initiatives had delivered the expected return over the last few years, and 16% had scaled across the enterprise.¹ A board that has read numbers like these wants evidence from your team, in units it already uses.

Expect the question in some version of these four:

1. What are we getting for the AI spend?
2. Are we shipping more of what we agreed to build, or only more?
3. Did quality or reliability suffer?
4. What will you change next quarter, and what will it cost?

Each one has a measurable answer, and none of the answers is an activity count. Leverage and the outcomes the shipped work moved answer the first together. The share of finished work that traces to an agreed intent answers the second, and delivery stability answers the third. The fourth comes from the diagnostics that explain leverage, which show where the team’s attention goes.

Velocity, story points, pull request counts and lines of code all count activity, and agents produce activity cheaply. Work that merges and stays merged is still engineering output. Value arrives when that work reaches users and moves the customer or business outcome the team agreed to target.

Pull request counts show the gap most clearly. Faros looked at telemetry from more than 10,000 developers. On teams with high AI adoption, developers merged 98% more pull requests, while time spent in review rose 91%, average pull request size grew 154%, and bugs per developer rose 9%. Faros found no significant link between AI adoption and delivery outcomes at the company level.²

The most quoted AI number in public is the share of code a model wrote. On Alphabet’s third-quarter 2024 earnings call, Sundar Pichai said “more than a quarter of all new code at Google is generated by AI, then reviewed and accepted by engineers.”³ A share-of-code figure says how much code a model wrote. It says nothing about whether anyone needed that code, or what reviewing it cost.

Suggestion acceptance rate has the same weakness. In GitHub’s study of its own code completion tool, the acceptance rate of shown suggestions was the best predictor of how productive developers felt.⁴ It predicts perception, which is a different thing from output.

Perception is unreliable on its own. In METR’s randomized trial, experienced open-source developers took 19% longer to finish tasks with early-2025 AI tools. Afterward, they believed the tools had made them 20% faster.⁵ METR’s 2026 update estimated an 18% speedup for the developers who returned, with a confidence interval running from a 38% speedup to a 9% slowdown, wide enough to include no effect.⁶

Every one of these numbers also bends once it becomes a target. Charles Goodhart observed in 1975 that “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”⁷ Marilyn Strathern’s later phrasing is the one most people remember: “When a measure becomes a target, it ceases to be a good measure.”⁸ A team asked for more pull requests will open more pull requests.

Both say to measure the delivery system with several numbers read together, and both warn against a single activity count. Leverage fits inside them, with DORA’s stability metrics as its guardrail.

DORA measures software delivery with five metrics.⁹ Three describe throughput: change lead time, deployment frequency, and failed deployment recovery time. Two describe instability: change fail rate and deployment rework rate, which DORA defines as “the ratio of deployments that are unplanned but happen as a result of an incident in production.”

DORA’s surveys show why the stability side matters. Its 2024 report estimated that every 25% increase in AI adoption came with a 1.5% reduction in delivery throughput and a 7.2% reduction in delivery stability.¹⁰ By 2025 the throughput relationship had turned positive, while higher adoption still went with more delivery instability. The same report found that 90% of respondents use AI at work and 30% trust AI-generated code a little or not at all.¹¹

The SPACE framework names five dimensions of developer productivity: satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. Its authors wrote that developer productivity “cannot be measured by a single metric or dimension.”¹²

Leverage respects both warnings. It puts output (performance) over attention (efficiency and flow), counts review of other people’s work (collaboration), and ignores raw activity. It does not replace DORA’s stability metrics, which belong beside it on every report.

The time moved. Agents shorten the writing of code, and the people around that code absorb the extra volume in review and rework, which most dashboards never count.

Controlled studies do find faster tasks. Google ran a randomized trial with 96 of its own engineers. Those with AI features finished an enterprise coding task about 21% faster, with a wide confidence interval.¹³ Field experiments at Microsoft, Accenture, and a Fortune 100 company found a 26.08% increase in completed tasks among developers given GitHub Copilot.¹⁴

The cost lands after the task. GitClear found that code churn, the share of new lines revised within two weeks of being written, rose from 3.1% in 2020 to 5.7% in 2024.¹⁵ Two in three developers say their biggest frustration with AI tools is output that is “almost right, but not quite,” and 45% say debugging AI-generated code takes more time.¹⁶ CodeRabbit’s review of open-source pull requests found about 1.7 times as many issues in AI-authored pull requests as in human-written ones.¹⁷

Rework was expensive before agents. Boehm and Basili’s widely cited estimate put avoidable rework at 40 to 50% of a software project’s effort.¹⁸ Agents make rework faster to produce and easier to miss, because the fix arrives as another pull request that looks like output.

So a measure of AI’s impact has to count review and rework as cost. Leverage does, because both are attention. Work that bounces back costs twice: the finished work arrives later, and the fix adds hours to the denominator.

Track leverage, the finished work the team ships per hour of attention, and read it monthly beside the customer outcomes the work targeted and a few diagnostics that explain why it moved. Retire story points and velocity as evidence of AI’s impact, because agents inflate both without changing what ships.

Story points estimate effort, and agents change the effort behind a point from one week to the next, so velocity drifts with the tools. Leverage keeps the size of the work fixed at what it would have cost the team without agents, and puts the hours people actually spent beside it.

Leverage and the outcome results answer the board’s first question together. Leverage says how much finished work each hour of attention bought, and the outcomes say whether that work changed anything for customers or the business. The diagnostics explain it, and each one points at a different place in the work. Ship-ready rate and rework by cause show whether agents build what was meant. Attention by phase shows where people’s hours go, including the review a manager does for everyone else. Agreed workstreams in flight shows how much work runs at once from shared intent. Agents add capacity there, but parallel agents raise leverage only when their work merges without pulling people back in.

The guardrails keep the headline honest. A team can raise leverage for a month by skipping review, and the change fail rate will show it. Spend belongs beside output as a cost, never as a measure of progress.

List the work that merged in the last 30 days and drop anything that came back. Size each item by what it would have cost without agents, add up the hours of attention the team spent, and divide the first total by the second.

Sizing is the weak point, so make it boring. Estimate each piece of work before agents start, in the hours the team would have needed by hand, and use the same method every month. A team that estimates generously will inflate its own number. Size a few pieces of work the team built without agents and compare, so the scale stays honest.

Attention can be captured in one of two ways. The first infers it from timestamps the team already produces: messages to agent sessions, pull request reviews and comments, commits, and edits to intent documents. Group each person’s events into sessions, join events less than 20 minutes apart, add 5 minutes to each session, and count a lone event as 5 minutes. The second is a weekly log of five numbers per person, one per phase. A log is an estimate, so keep the method fixed and read the trend, not the level.

Charge rework to the work that caused it. A pull request reverted within 30 days of merge stops counting as finished work, and the hours spent on it stay in the denominator. A follow-up fix charges its hours to the original work. Speed that breaks things later then shows up on its own ledger, without anyone labeling it bad.

Rework avoided falls out of the same records. Take the rework hours per hour of finished work in your baseline, subtract this month’s rate, and multiply by this month’s finished work. The result is hours the team did not spend fixing, and it is the most concrete number you can give a CFO.

From three places: your git host, a record of agreed intent, and a label on every pull request that came back. Most teams have the first, few have the second, and the third takes a week to start.

The rework labels matter more than they look. Rework caused by intent is the cheapest to avoid, because the fix happens before any agent starts: agree on what to build, by name, and record the decisions behind it. Before the agents start describes that step. Defects and scope changes have other causes, and mixing the three hides which one is growing.

In Hamster, intent and its agreement already live in one record. Each Brief carries the Ready and Not yet votes of the people whose work it changes, the Plan that agents build from, and the pull requests linked to it. The customer outcomes sit on the same chain, because each Brief belongs to an Initiative that links to the Goals it serves. Leverage can be tracked as a Goal whose Metric is tagged leading or lagging, with a baseline, a target, and a Result logged each period.

One headline number with its trend, the customer outcomes the shipped work moved, the diagnostics that explain the number, the risks, and one change for next month. Keep the same order every month, so the board learns to read it.

The last line earns the board’s trust. A leverage figure built on half the team’s hours is still useful, as long as the board knows it covers half. State the coverage and the sizing method every time, and change neither without saying so.

Name real work in “What shipped and what it moved.” If a workstream shipped and its outcome has not moved yet, say so and give the date you expect to read it. A board remembers the workstream that moved a company goal far longer than a ratio. The link between the two shows that the AI spend paid for something the business wanted. In Hamster, a scheduled Routine can draft this page on the same day each month from the Briefs and Plans that shipped, for you to check and send.

Spend the first month on a baseline, the second on a first honest reading, and the third on the first report. Change nothing because of a single month.

The baseline is the step teams skip, and without it every later number is an anecdote. Ninety days of merged pull requests, with reverts and follow-up fixes marked, is enough to compute a first leverage figure and a first ship-ready rate. It also shows how much rework the team carried before anything changed.

The worst mistake is turning the scorecard into a target for individuals. The rest come from reading too little data, or from changing how it is counted.

• Do not rank people. Reviewing other people’s work counts as attention, so it lowers the reviewer’s own figure on purpose, which makes a manager’s review load visible. A ranking would punish the people who review, so report the team and show each person only their own figure.
• Do not set targets on the diagnostics. A target on ship-ready rate invites smaller, safer work, and a target on alignment time invites agreement nobody means. Set the target on leverage, and use the diagnostics to find what to change.
• Keep the sizing method fixed for the quarter. A new method breaks the trend, so change it at a quarter boundary and restate the baseline.
• Wait before reading the latest month. Recent work keeps collecting follow-up fixes for 30 days, so each month settles over the next one.
• Leave out work that came back. A reverted pull request is not finished work, however fast it merged.
• State the coverage. If the method captured 20 of 40 working hours, the board should know.
• Describe what moved together, not what caused what. Most studies in this guide report associations, and your own before-and-after comparison is one too.
• Do not report leverage without outcomes. A rising number with no customer or business result behind it is still activity.
• Keep instability on the page. Leverage without the change fail rate invites the question you least want from a board.

A team that runs many agents at once is running a software factory, and Hamster makes that factory build what the team agreed before it builds anything. The team shapes intent together in a Brief, votes Ready or Not yet on it, and hands the agreed version to agents through a Plan. Agents in Claude Code, Cursor, and Codex read the Brief and the team’s Context Graph and write their decisions back while they work, and pull requests link back to the Brief they came from.

That record makes leverage measurable, because every piece of finished work traces to an intent people agreed to. Every review and fix attaches to the work it served, and every decision is kept with who made it. The same record keeps the outcome beside the number. Each Brief belongs to an Initiative, each Initiative links to the Goals it serves, and each Goal’s Results record the target and the actual every period.

You can run everything in this guide with a spreadsheet and your git host. If you want the record to stay live across every person and every agent session, Hamster was built for that.