AI knows its limitations. Do we know ours?
AI can give an answer in seconds, and too often it sounds really good on the surface. The question I keep asking: In the age of abundance, how can we have so much data and so little insight? I’ll share two stories that may surprise you, one of them my own mistake.
A few months ago, I was reading the results of a marketing conversion study. Markets had been slow to respond, so I expected soft numbers. Yet, the results were better than expected.
Based on the marketing goals and assumptions behind the measurement, my read was that travelers were booking on price at the last minute. Probably repeat travelers, too. Some undecided.
Someone on the team tested the same report with an AI tool. Here is what the chatbot said:
Stay in market all year. Run search, paid social, and more, this and that. Build new pages. Capture emails. Hire influencers. Add PR.
It laid out a full-funnel media plan. And it sounded really good. However, it had almost nothing to do with the data presented. It also made the recommendations without enough context on the brand, marketing budget, current market conditions, or internal team insights.
The reality is that anyone could have gotten that answer using a similar tool.
The answer was too generic.
If you swap in any other travel brand or destination, you’ll probably get a similar response.
It is tempting to blame the tool. I would not. The chatbot did what it was asked. It produced results with little or no context. It did not know the markets had been slow. It didn’t know who usually travels there or what the budget was. It knew very little, so it filled the gap with the most common playbook it had. It is known as the ‘sea of sameness’ in marketing. With results analysis so readily available, we are becoming increasingly data-driven, trusting numbers too much while sacrificing wisdom and deeper insight.
A lot of people now use AI to read a data report for them. What went up, what went down, and by how much. In fact, that is never the hard part. Anyone can read a percentage.
The hard part is the why.
Why this result?
Why these travelers?
Why now?
That part lives with the people who understand the strategy and know the product well.
Good critical thinking helps you decide what to do with AI’s answer.
The idea here is that the more people trust the AI, the less critical thinking they apply to its answers. The more people trust their own expertise and lived experiences, the more they check AI’s work.
A KPMG and University of Texas at Austin study illustrates this tension. The research of early-career professionals revealed that those who did well using AI were the ones who first framed the problem, set standards, and questioned the AI’s output. In this context, critical thinking and AI fluency are like muscles. You have to keep flexing them. Judgment stays sharp only through use.
Ask three questions before anything goes out.
I now ask three questions of any AI output.
01
Blind test
What do I know about these numbers that the report does not?
The tool sees the number. You, on the other hand, have insights through your relationships and understanding of the markets. The hotel closed for renovation. The event date changed, skewing the month. If what you know would change the story, the AI’s story is incomplete.
I call this the blind test. Before you start a task, write down your own reasoning. What did you expect, and why? Then, use the tool. Is the result good enough? What do you know that the report does not show? If the results differ, find out why. Usually, the tool is missing something only you know about.
Is this insight specific to our situation, or could it apply to any destination?
02
Any-destination test
Could this exact insight/recommendation sit in any other destination’s plan?
I call this one the any-destination test. If it fits any destination, it fits none. The always-on plan failed here first. “Be in every channel” could fit almost any plan, but was it the right move for this market?
Bharat Anand and Andy Wu explain why this matters in HBR: “Everyone has access to gen AI.” When competitors use similar tools for similar tasks, the gains do not stay with any of them. Every destination chasing the same traveler can open the same tool. The advantage comes from what you bring to it: human insight. So give the AI your read, and ask it to argue back and question your assumptions. You’ll be surprised.
03
Board test
What happens if this is wrong and nobody catches it?
Not every AI output needs the same level of oversight. I call this the board test. Anand and Wu sort tasks with two questions. How costly is an error? Does the task run on explicit data, or on judgment? A meeting summary can be skimmed by an internal team. A data table sent to your board requires checking every figure.
A recommendation on the marketing budget is different. AI can already reallocate media spend between ads and audiences on a demand-side platform (DSP). Deciding where the budget should go as strategy shifts is a leadership call. It needs your own understanding of the markets and curious minds at the table.
Speed is great and needed, but it is not all progress.
Level of human oversight
Skim
A meeting summary for the internal team
Check every figure
A data table sent to your board
Leadership call
Where the marketing budget should go
The model did what it was asked.
When an AI project disappoints, the response is to blame the model or use a smarter one.
Two years ago, I was rushing to finish a draft before an overseas business trip. I pulled together my notes and my rough draft ideation. I call it the brain dump. I fed it into a writing chatbot I had built. The system instructions were based on my writings, publications, and writing style.
I didn’t open what came back because it had been producing solid outputs. I sent it straight to Kris on my team to polish before boarding my flight. Then I got on the flight.
When I returned, I opened Kris’s version. It sounded nothing like me. I was confused. Because Kris is so good.
I asked Kris what happened. She pointed me to the document I had sent her.
I opened it. I was terrified.
The chatbot had failed me. Hallucinations were more common then.
But I was the one who skipped the steps.
I never opened the output. And nothing at the top of the document said it was an AI draft. Kris had no reason to treat it as anything but mine.
It was the best lesson I have learned. Now I open and check every AI output before it goes anywhere. We also developed internal evaluations where an agent can review another agent’s work. An AI draft also says at the top which tools it used, until a person makes it their own.
Tools used: our SCAN research agent, Grammarly
No AI could have saved me. Opening the file would have. Whether or not AI helped, the work product is yours. You have to stand behind every word. I learned that one the hard way.
Back to that conversion study. Imagine if someone had taken the AI’s recommendations at face value. Because AI said so, they would have funneled more money into more channels, aimed at travelers who were already booking on price at the last minute. The market was still worth investing in, just not in everything at once. The data was telling a different story. Someone just had to read it.
This is where culture matters.
Everyone on your team should feel they have permission to think out loud and ask questions. An AI label at the top of a draft only works if people feel safe using it. No one should feel embarrassed saying, “AI wrote the first pass.” Build openness in how you respond. Stopping at what did not work shuts people down.
“That doesn’t work, but what if we try this?” invites them back in.
One thing worth trying, if it fits your team: labeling AI outputs. Then ask questions about them:
Is this good enough? How can we close the gaps in AI-assisted tasks? These are learning questions.
Leaders set the tone.
When you ask these questions out loud, including about your own work, you give people space to try, test, learn, refine, and share what they find.
AI is a human technology. It can give you an answer in seconds. What you bring to that answer is the art. The human touch with intention and purpose.
“The real danger is not that computers will begin to think like men, but that men will begin to think like computers.”
— Sydney J. Harris
*********
