KPIs and the Three-Tier ROI Stack

Start with the uncomfortable fact every leader should carry into the boardroom: adoption is near-universal, but measured value is rare. The differentiator between the winners and everyone else is not the model — it is whether leadership measures the right things and acts on them.

The 2025–2026 evidence converges hard:

"Everyone has AI; almost nobody has AI value yet."

The gap is not a technology problem — it is a measurement-and-management problem. When pilots don't move EBIT, the cause is rarely the model; it is the absence of a baseline, a workflow redesign, and a disciplined number to hit. McKinsey's own analysis found that tracking well-defined KPIs is the single attribute most correlated with AI adoption — measurement is not the scoreboard after the game, it is part of how you win. These are time-stamped findings from a fast-moving field; treat the percentages as directional and re-verify them live before you quote them.

The core framework of this lesson is a simple ladder. AI value reveals itself in three tiers, and you must climb all three. Most organizations proudly report tier one and never get to tier three — which is exactly why their AI looks busy but their P&L doesn't move.

The discipline is to connect the tiers, not to stop at one. Adoption without efficiency is just a tool nobody benefits from. Efficiency without P&L impact is a productivity claim with no proof it reached the bottom line — time "saved" that is never reinvested or removed from cost is value that evaporates. The leadership move is to define, before you spend, what a win looks like at tier three and trace the line back down to the tier-one behaviors that produce it.

A caution from the research: be deliberate about where you chase tier-three value. MIT NANDA found more than half of gen-AI budgets flow to sales and marketing tools, while some of the largest near-term ROI sat in unglamorous back-office automation. The flashiest use case is not always the one that moves the most money.

Some metrics are not just weak — they are actively misleading, because they can move in the opposite direction of value. These are the numbers a vendor or an over-eager team will put on a slide precisely because they always go up.

The most counter-intuitive trap is that these can be inversely correlated with value. Consider agentic AI: as work moves from one-human-one-seat toward one orchestrator running many agents, a more valuable, more automated workflow may show fewer seats and fewer human logins. If "seats" or "logins" is your success metric, your dashboard will fall exactly as your value rises. You would be punishing your best transformation.

The fix is to anchor every vanity metric to a tier-two or tier-three outcome. Seats only matter if utilization converts to time saved; time saved only matters if it converts to cost out or revenue in. Treat tier-one metrics as diagnostics (is anyone using this?) — never as the scoreboard.

Two disciplines separate credible measurement from theater: baseline before you start, and track quality, not just volume.

Baseline before you spend. If you don't capture today's handling time, resolution rate, error rate, CSAT, and cost before you turn AI on, you can never prove the change was AI's doing. "Vibe-based" spending — funding AI on fear-of-missing-out with no baseline and no hypothesis — is one of the most-cited reasons ROI goes unproven (Worklytics, 2025). A pre-registered baseline turns a debate about whether AI "feels" helpful into a measurable before-and-after.

Track quality, not just volume. Volume metrics (chats handled, tickets closed, content generated) are seductive because AI makes them spike. But volume can mask a quality collapse — and quality is what customers and revenue actually respond to. Always pair every volume metric with a quality metric: containment/deflection alongside CSAT/NPS, throughput alongside error/defect rate, speed alongside accuracy. This is the Klarna lesson (next callout), and it is the difference between a number that flatters you and a number that protects you.

At the top of the stack sits the metric that separates serious programs from busy ones: the percent of EBIT attributable to AI. McKinsey uses precisely this line to identify high performers — the firms that can attribute a meaningful, growing share of earnings to AI, rather than gesturing at productivity anecdotes. As of late 2025, even among the ~39% of firms reporting any enterprise EBIT impact, most attribute under 5% of EBIT to AI (McKinsey, State of AI, Nov 2025) — which tells you both how early this is and how distinctive it is to be on the right side of that line.

Set a realistic clock. AI returns take longer than typical technology investments. Most organizations reach satisfactory ROI in roughly 2–4 years; only about 6% see it in under a year (Deloitte, AI ROI Paradox). Promising board-level EBIT impact in two quarters is how you manufacture the disappointment that gets a good program cancelled. The leadership move is to commit to the 2–4-year arc on tier three while demanding near-term movement on tiers one and two as leading indicators — proof you're on the path even before the EBIT lands.

Both the EBIT-share figure and the 2–4-year timeline are volatile, dated findings. Teach the durable concept — EBIT attribution is the high-performer line; AI ROI is a multi-year arc — and re-verify the exact numbers at the live sources before you cite them.

Measurement without consequence is decoration. The hardest — and most valuable — leadership act is the scaling decision: at the end of a pilot, every initiative either scales, gets reshaped, or gets killed. What you must not allow is pilot purgatory — the endless proof-of-concept that never faces a verdict. The share of companies abandoning most AI projects rose to roughly 42% in 2025, up from ~17% in 2024 (cited via Worklytics/S&P; trace to primary before quoting); the lesson is not to avoid killing pilots — it is to kill them deliberately and early, on evidence, rather than letting them drift.

Use a simple decision rule tied directly to the three-tier stack:

The positive counter-example to pilot purgatory is JPMorgan, which runs 450+ AI use cases in production with benefits growing ~30–40% year over year — the hallmark of scoped, KPI-anchored deployment where each use case has to earn its place against a number. The contrast with the ~95% of pilots showing no P&L return is not luck; it is the discipline of measuring, then deciding.

You don't need to build the dashboard — you need to interrogate it. These are the questions that separate a measured program from a flattering one. Keep them on a card for every AI review.

On any metric you're shown:

• Which tier is this — adoption, efficiency, or P&L? (If everything on the slide is tier one, push back.)
• Could this number go up while value goes down? (The vanity-metric test.)
• What baseline is it measured against? Did we capture "before"?

On quality and risk:

• For every volume metric, where is its paired quality metric (CSAT/NPS, error rate, accuracy)?
• If this is "90% accurate," accurate how — and who owns the failing 10%?

On the P&L line:

• What % of EBIT do we attribute to AI today, and what's the trajectory?
• Is our timeline honest — are we judging a 2–4-year payback on a two-quarter clock?

On the verdict:

• Which pilots haven't moved a number, and when is their kill-or-reshape decision?
• When we report time saved, has it actually been reinvested or removed from cost — or did it just evaporate?

Asking these consistently does something subtle but powerful: it trains the organization to bring you tier-three thinking by default, because they know tier-one theater won't survive the meeting.