Interrogating an AI Claim

Most AI skills a leader needs are about adoption — picking use cases, funding pilots, redesigning workflows. This one is different. It is a skill of resistance: the discipline to not believe a confident machine.

Here is the situation you will face, over and over, in board decks and vendor pitches: a system is described as "90% accurate." It sounds reassuring. It sounds like a fact. And the natural executive instinct — built over a career of trusting credible-sounding numbers from credible-sounding people — is to nod and move on.

That instinct is exactly the trap. "90% accurate" is not an answer; it is the beginning of a conversation most leaders never have. The number tells you nothing about what kind of accuracy was measured, what the failing 10% actually look like, or who is on the hook when one of those failures reaches a customer, a regulator, or a court.

The danger with today's AI is not that it is weak. It is that it is silently and invisibly bad in some places — and it never tells you which places those are.

Research makes this concrete. In a 2023 field experiment by Dell'Acqua, Mollick, Lakhani and colleagues (Harvard/BCG, 758 BCG consultants), AI users on tasks inside its capability were roughly 25% faster and produced ~40% higher-quality work. But on tasks that fell outside that capability, AI users were about 19 percentage points less likely to reach the correct answer than colleagues working without AI at all. Same tool, same confident tone — opposite outcomes. The tool gave no warning when it crossed the line.

Learning to interrogate the claim is how you stop being the executive who trusts the wrong 19 percentage points.

Memorize one move. When anyone — a vendor, a team, a slide — tells you an AI system is "X% accurate," respond with three linked questions:

"Accurate in what way? What do the failing 10% actually look like? And who is accountable for them?"

Each question opens a door the headline number was holding shut.

The second question is the one that separates seasoned leaders from the rest. The shape of the errors matters more than the headline number. A 95%-accurate system whose 5% of failures all land on your most vulnerable customers is worse than a 90%-accurate system that fails randomly on trivia.

The third question is the one that protects your enterprise. A model is only the front half of the sentence; the back half is "…and a named human checks it before it does anything that matters." If no one can answer who owns the 10%, you have not bought a capability — you have bought an unowned liability.

Why is this skill so hard? Because you are fighting a well-documented cognitive pull called automation bias — the human tendency to over-trust a confident, automated output and under-apply your own scrutiny, simply because a machine produced it.

Automation bias is sneaky precisely because it feels like efficiency. A polished answer arrives instantly, formatted, fluent, and self-assured. Your brain reads that fluency as competence and quietly lowers its guard. The better the output looks, the less you check it — which is exactly backwards, because today's models are most dangerous when they are most fluent and most wrong.

Three conditions make automation bias most damaging. When you see them stacking up, raise — don't lower — your scrutiny:

• High confidence in the output. The model never says "I don't know." It states a fabricated answer in the same authoritative tone as a correct one.
• High stakes in the decision. The cost of a wrong answer is large (legal, financial, medical, safety, reputational).
• Low human friction. The output flows straight into action with no human checkpoint, so no one is positioned to catch the error.

The antidote is not distrust of all AI — that wastes the real value. It is calibrated trust: trusting AI more where it is provably good and the stakes are low, and trusting it less exactly where its confidence is highest and a wrong answer is load-bearing. Your job as a leader is to design that friction back in at the high-stakes points your teams, under deadline pressure, will be tempted to remove.

Here is the single most useful operating principle for an executive working with AI on anything that matters:

Use AI to expand your options, not to make your choice.

AI is extraordinary at the divergent half of thinking — generating possibilities, surfacing considerations you missed, drafting three versions of a plan, stress-testing an argument, summarizing what others have done. That is genuine leverage, and you should use it aggressively. Where it fails is the convergent half — weighing trade-offs against your context, your stakeholders, your risk appetite, and your accountability. That is judgment, and judgment is yours.

The practical test is a question you ask of your own workflow: am I using this AI to widen the funnel, or to close it?

The left column makes you a better decision-maker by giving you more and better raw material to judge. The right column quietly makes the decision for you while leaving your name on it — the worst of both worlds, because you carry the accountability without having exercised the judgment.

This maps cleanly onto the divergent/convergent split: let AI run wide; you run deep. It produces breadth; you provide the weighing, the context, and the call.

Two specific failure modes deserve a leader's particular vigilance, because they are most likely to ambush you precisely on the high-stakes, strategic questions where you most want help.

1. Generic strategy advice ('trendslop'). When researchers asked large language models for strategic advice, they got back what HBR (2026) memorably labeled "trendslop" — fluent, confident, on-trend, and almost entirely generic. The advice pattern-matches to whatever sounds current ("embrace digital transformation," "build an AI-first culture," "focus on customer-centricity") without engaging the one thing that makes strategy strategy: your specific context, constraints, and competitive position. It reads like a McKinsey deck and commits to nothing. The tell: advice that would apply, word-for-word, to any company in your industry — which means it is advice for no company in particular.

2. Flattery. The same research stream finds these systems can subtly flatter and over-agree with the user. Trained to be helpful and agreeable, a model will too often tell you your idea is strong, your analysis is sharp, and your instinct is right — because agreement is the path of least resistance, not because it has independently judged your idea to be good. For an executive, an assistant that reflexively validates you is not an asset; it is an echo chamber with a confident voice.

Together these explain why AI is worst at exactly the work leaders most wish to delegate: genuine, novel strategic reasoning. It will hand you a polished, flattering, generic plan and feel like help — which is more dangerous than obvious nonsense, because polish disarms scrutiny.

Everything in this lesson resolves to one non-negotiable principle: automation for execution, humans for judgment. AI can draft, summarize, sort, surface, and propose. It cannot be accountable — and accountability is the irreducibly human part of leadership.

This is not a soft preference; it is increasingly a legal and governance reality. Three signals every leader should register:

• The law is closing the accountability gap. In Mobley v. Workday, a US court allowed a nationwide age-discrimination collective action to proceed and ruled that an AI screening tool can be treated as the employer's "agent" — meaning you can be held liable for what your AI does. "The AI decided" is not a defense.
• "The AI did it" is not a shield, it is an exposure. When AI-fabricated content reaches the real world, the human who relied on it pays. By early 2026 there were well over 1,300 documented cases worldwide of AI-hallucinated legal citations reaching court filings — including from elite firms — with lawyers sanctioned for output their AI confidently invented.
• Governance frameworks demand a named owner. Every serious AI governance framework (NIST's AI Risk Management Framework, ISO/IEC 42001, the EU AI Act) converges on the same requirement: every production AI system has a named human owner accountable for its behavior, and high-stakes decisions retain meaningful human oversight.

So the closing discipline is simple to state and hard to hold under deadline pressure: AI expands your options and accelerates your work; you make the call, you can explain it, and your name is on it. Keep a human in the loop wherever a wrong answer is load-bearing, and never let the speed of a confident machine talk you out of the judgment only you can supply.