The Right Questions to Ask

A non-technical executive will never out-argue a vendor on model architecture, and shouldn't try. The leadership skill is different and more durable: asking the question that makes the truth surface on its own. A confident demo, a slide that says "90% accurate," a roadmap full of "agentic" — none of these survive contact with the right follow-up. Your job is to ask it.

This matters because the evidence says most AI value is lost not in the lab but in the boardroom and the meeting room — in decisions made on a polished surface without interrogating what's underneath. Recall the headline numbers from this course: roughly 95% of enterprise generative-AI pilots delivered no measurable P&L return (MIT NANDA, State of AI in Business 2025), and McKinsey's State of AI 2025 found that about 51% of organizations had at least one negative AI incident in the prior year. Behind most of those is a question that nobody asked.

The questions in this lesson are organized on five fronts, mirroring the five places leaders are most often misled:

Use AI — and use these questions — to expand your options, not to make your choice. Final judgment and accountability stay human.

Every AI conversation eventually produces a number — "90% accurate," "cuts handling time by 40%," "resolves two-thirds of tickets." Treating that number as a fact is the most common, and most costly, mistake a leader makes. A headline metric is the beginning of a conversation, not the end of one.

The interrogate-the-claim triad breaks any number open:

"Accurate in what way — what do the failing cases look like — and who is accountable for them?"

• "Accurate in what way?" Accuracy on which task, measured against what benchmark, on data that looks like ours? A model can be 90% accurate on a vendor's curated test set and far worse on your messy, real-world inputs — the jagged-frontier problem, where capability is unevenly good and the boundary is invisible.
• "What do the failing cases look like?" A 10% error rate spread evenly across low-stakes cases is a different animal from a 10% rate concentrated on your highest-value, highest-risk decisions. The shape of the failure matters more than its size.
• "Who owns the failures?" When the model is wrong, who catches it, who is accountable, and what does it cost? If the answer is "nobody, until a customer complains," you have found your real risk.

This is the antidote to automation bias — the human tendency to over-trust confident-sounding output. Remember the dossier's example of generic AI strategy advice (HBR's "trendslop", 2026): fluent, plausible, and hollow. Confidence is not correctness.

The strategy questions decide whether your AI investment produces a step change or a rounding error. The dossier is blunt on this: there is no standalone "AI strategy" — only a business strategy strengthened by AI. The leaders who make "doing AI" the goal generate diffuse pilots with no P&L impact.

Three questions cut to the heart of it:

1. "Where can we move fast, and where must we prioritize control?" Not every use case carries the same risk. A marketing-copy drafter and a loan-approval model belong on opposite ends of the speed-vs-control dial. Ask which dial position each initiative sits at — and whether anyone has consciously chosen it.
2. "Are we using AI to expand our options, or to make the decision for us?" The right posture is AI as an option-generator that a human judges, not an oracle that a human rubber-stamps. The moment a system is making the choice rather than widening the menu, accountability has quietly slipped.
3. "Is this a tool rollout or a workflow redesign?" This is the single most important strategy question, because it predicts whether value will materialize.

That third question rests on the strongest finding in the research. McKinsey's State of AI 2025 found workflow redesign has the single biggest effect on the EBIT impact an organization gets from AI — yet most firms simply bolt AI onto legacy processes. This is the gen AI paradox: ~80% of firms use generative AI, yet ~80% report no material bottom-line impact, because value sits in deep, redesigned vertical workflows, not shallow horizontal copilots. A copilot makes an existing task faster (incremental); an agentic, redesigned workflow does the task end-to-end (a step change). If the plan in front of you is "give everyone a copilot," you are buying speed, not transformation.

The decisive adoption question is deceptively simple:

"Are we empowering our line managers to make this work — or are we hoping a central lab will fix adoption for us?"

The research points hard at the first half. The executive-versus-manager reality gap is one of the most under-appreciated reasons pilots stall: senior leaders see strategic advantage, while middle managers confront AI's flaws in real workflows, often unsupported (HBR, 2026). When executives mandate adoption top-down and a central team "owns" it, the people who actually have to change how their teams work are left out of the design — and quietly resist.

The operating-model answer the evidence favors is hub-and-spoke: a lean central hub sets standards, guardrails, and platforms, while business-unit "spokes" own execution and adoption locally. Firms that successfully scale AI are markedly more likely to use this structure than a fully centralized or fully decentralized model (Dataiku/IBM, cited in the dossier). The central lab is an enabler, not a gatekeeper — and never a substitute for engaged managers.

A short adoption-diagnostic question set:

That last row matters more than it looks: BCG's AI Radar found deeply engaged C-suite teams are roughly 12× more likely to be top-tier AI value creators. Championing is not delegation — it is role-modeling.

Every AI ambition eventually runs into the same wall, so ask about it before you fund the ambition:

"Is our data clean, governed, and accessible enough to feed this — honestly?"

Data is the one AI advantage that compounds and the one rivals cannot simply buy. But the same data is a liability when it is siloed and ungoverned, because — as the dossier puts it — "AI agents cannot query what they cannot find." Roughly 80% of the real work in an AI initiative is data, governance, and workflow integration, not the model (MIT Sloan, via the dossier). The model is the easy 20%.

Break the data question into its three load-bearing words:

• Clean — Is the data accurate, current, and consistent, or will "garbage in, garbage out" surface as confident, wrong answers? Machine learning learns from data, so data quality is the literal raw material.
• Governed — Do we know what data this system can access, where it flows (including to third-party model providers), and who approved it? This is the first risk-taxonomy question every board must be able to ask.
• Accessible — Can the system actually reach the right data when it needs it, or is it locked in silos and legacy systems? Eight in ten organizations cite data and legacy-integration limits as the gate on scaling agents.

The payoff for getting this right is the data flywheel: every interaction generates feedback data that makes the system better, which compounds into a moat. But a flywheel built on dirty, ungoverned, inaccessible data just spins faster in the wrong direction.

When a system doesn't just answer but acts — books, pays, emails, approves, transacts — the questions change in kind, not just degree. An agent that takes actions toward a goal needs governance precisely because it does things. The two non-negotiable questions:

"What is our human-in-the-loop policy for this system — and can we halt it and roll it back?"

Ground this in the autonomy delegation test: delegate a decision to a system only when the data is clear, the rules are defined, and the cost of waiting for a human is greater than the cost of a wrong autonomous action. The governing principle is automation for execution, humans for judgment. Autonomy is a dial, not a switch — and that single dial sets the value, the risk, and the governance required.

The agent question set every leader should carry:

The security dimension is real and board-relevant. Prompt injection — hidden instructions buried in a document, email, or web page the agent reads — can hijack an agent with tool access into exfiltrating data or making unauthorized transactions (OWASP Top 10 for Agentic Applications, 2026). The mitigations are conceptual but askable: least-privilege tool access, sandboxing, input validation, and human approval for high-impact actions. And the liability is not hypothetical — in Mobley v. Workday, a court treated an AI screening tool as the employer's "agent," a precedent that shifts responsibility for what the software does back onto the organization deploying it.

Pull the five fronts together into a single page you can carry into any vendor pitch, pilot review, or strategy meeting. Lead with the front that matters most for the decision in front of you.

Three cross-cutting questions belong on every front, because they expose the answers people most want to avoid giving:

• "In P&L terms, not demo terms — how will we know this worked?"
• "Are we buying or building, and why?" (The dossier's evidence favors a default-to-buy posture for most use cases — verify the specific figures live.)
• "Who is the single named owner accountable for this — and can they explain it to a regulator?"

A question bank is only as good as the discipline of using it. The habit to build: never approve an AI investment, sign a vendor, or green-light a pilot until each relevant front has a real answer — not a confident deflection.