Holding back is a skill.
What data practitioners say about AI at work.
As we learn how AI works, we are all trying to figure out two things: when to rely on it and when to hold back. And as we better understand this human technology, the second question matters more.
When to hold back when using AI became the starting point for a deeper understanding of where human expertise still matters most. To explore it, we went to the data + research people who navigate that tension every day. Plus, data people are always pragmatic.
Our goal was to understand how data professionals actually discuss their experiences of AI at work with their peers. We analyzed public online conversations and forums among data analysts, business intelligence specialists, data scientists, and researchers in the summer of 2026.
We found six patterns across every community:
Humans are still in the loop, and AI is doing the busywork.
The bottleneck is the data foundation, messy but necessary work.
Humans define intent, and AI executes.
Fluency matters more than adoption.
Domain experts are golden.
And the “jobs” worry is real, but it is about leadership decisions and talent, not work-force.
Most importantly, why do we still use the word "work-force" in the age of AI? The term dates to the 1940s, when it was coined for an industrial time of interchangeable labor. Isn't transformation about the people and talent?
Now, let’s get to the findings and what they mean.
Practitioners are attuned.
People embedded in the data world use AI every day, as reflected in the conversations. They are also watching it get oversold around them. In this sense, AI can’t do everything yet. The harshest discussions were aimed at colleagues who used AI foolishly. They came up with outputs and conclusions carelessly when the data didn’t support them. In this case, they were criticized for being lazy, for not having enough industry experience, or for not checking their work.
It is like the spoiled milk test. If it smells, it is likely spoiled.
Humans are still in the loop.
Many practitioners talked about the importance of human judgment in analysis. Most brought up the idea of AI as the analyst. In reality, practitioners use AI as an assistant. Drafting SQL queries. Generating first-pass code. Summarizing meeting notes. Writing documentation. Writing code.
A data scientist said:
“AI is best at the unsexy work. Boilerplate, documentation, repetitive queries. The things people get hired for, it’s worst at those”
A BI practitioner went further:
“The hard part of BI was never interpreting a chart. It was getting trusted data in the first place.”
The value of AI is still in streamlining analysis operations, according to the discussions. With AI, teams want to get to answers faster. Some of the interpretive work stays with people. “Tools sold as replacements for analytical judgment will likely fail. It is an overpromise at this point,” said one senior data professional. Tools that take on routine data work will likely provide better-than-expected results.
The bottleneck is the data foundation.
AI works in proportion to the cleanliness of the data it operates on.
Garbage in, garbage out, an expression used since 1957.
On a solid foundation, it produces useful outputs. On a messy one, it amplifies its faults. And the challenging part described in the discussion forums was that it is likely to sound very confident and knowledgeable.
A senior BI lead described these AI failures as:
“They’re data foundation failures wearing an AI costume.”
A data-quality practitioner also emphasized that the data needs to be clean. Period. There also needs to be a lot of it. A few hundred thousand records will not get you to a useful output.
One data engineer explained:
“Giving AI access to raw schemas on a test database gives garbage answers every time. What worked was building defined queries with explicit metrics, then telling the AI what each one was for in addition to company knowledge. The output results worked like magic.”
AI magnifies the underlying data issues. Multiple definitions of "visitor" or “customer” across systems create ambiguity that AI cannot resolve. AI wants clarity.
The unglamorous work is what still matters. Standardizing metrics. Documenting. Storing data properly. Building governance. We are all paying the tech debt we have owed for many years.
Humans define intent. AI executes.
Discussions noted that you need both approaches to achieve successful outcomes.
When utilized with raw data, AI produces confident, plausible answers that are often wrong. It does not know which definitions the business needs to prioritize. It does not know that "visitor" means one thing in border statistics and another in spend, survey, location, or credit-card data. It guesses in a language that sounds very authoritative.
One developer described the mechanics: "You build queries. You tell the AI what each metric means. Then it kinda works." When utilized with defined metrics, the failure mode improved. AI needs explicit definitions, named owners, and clear traceability. Then, there is no guessing.
Another practitioner offered a principle we heard:
“Deterministic work in code, probabilistic work in AI.”
This advice is part of the guardrails needed in the operational processes.
Fluency is what matters.
Several practitioners noted that tracking how many people use AI tools misses the point. Adoption is much more than the usage time or token counts.
One researcher said,
“The people losing credibility with AI are the ones using it without knowing where it breaks.”
A data analyst emphasized checking outputs against data, even though it sounds polished and stated,
“Otherwise, you’re being used by AI.”
Teams that move fast without verification habits risk producing errors. And those errors could end up in front of a stakeholder or a customer. This framing shifts the debate about AI from technology to risk and to how organizations define areas where AI fails.
An important habit for building an organization’s AI fluency is developing a shared understanding of how the technology works and does not work: where the tools perform well, where they break down, and where they fail.
Domain experts are golden.
Every discussion magnified the importance of domain experts.
AI is a lever on judgment.
People have institutional knowledge, and they know what matters, what to prioritize, and which numbers to trust.
A senior analyst who spent six years in retail before moving to tech:
“What AI compresses is the time between question and answer. Knowing what question to ask takes years of running the operation.”
A data leader working on AI strategy:
“The organizations that win with AI protect their domain experts. The ones that lose try to route around them.”
Brain trust is an asset. Failure comes from treating AI as a substitute. Some even talked about the companies laying off their employees and then rehiring them.
The “jobs” worry is about leadership decisions.
This is the nuanced one.
Practitioners are largely unafraid that an AI system will walk in and take an experienced analyst's job right away. Their fear is rather quiet and more specific. They worry that leaders will decide fewer people are needed because AI can generate designed outputs quickly. At times, in a few minutes.
Task displacement has turned into “headcount pressure,” a phrase used to describe the following pattern:
A team produces recurring dashboards and first-pass analyses faster. Executives see the speed. Roles begin to consolidate. Work shifts to adjacent functions using AI tools, workflows, or agentic systems, without the same level of expertise. Now, the team has become smaller, and it is delivering the same outputs.
Two pathways emerged amid concerns about task displacement.
The first is the junior employees. Entry-level work contains the most repeatable tasks. Think querying, reporting, and production tasks. That makes it the most automatable. It is also how junior analysts learn. When that work is automated away and nothing replaces it, the apprentice layer disappears with it.
One consequence raises an important talent development question: If junior employees increasingly rely on AI, how will they build the experience and judgment needed to evaluate its work? Today, that responsibility often falls to highly experienced employees. But what happens when those experts retire in five, ten, or fifteen years? Who will be qualified to review the AI's work?
The second is quality. Market researchers described leaders who often mistake quick summaries, comment clustering, or synthetic respondents for “real research.” One researcher warned that AI synthesis omits critical knowledge about the product, the audience, and the cultural context. The risk here is a lack of evidence quality in a polished format.
And some discussions noted that a widely held belief is that analysts who use AI well will replace those who don't. In this view, the talent risk is becoming relative. The risk is person-to-person, rather than a simple question of which roles survive.
AI is changing the production of intelligence. The leadership challenge is to redeploy people’s capacity toward better processes, higher quality, stronger evidence, and better decisions.
What to ask before an AI initiative?
Have we audited our data foundation and resolved issues around dysfunctions?
How do we define our business metrics in a way AI can actually use?
How do we measure adoption through the lens of fluency?
How are we using AI to support our domain experts?
What approach do we use in redesigning processes?
How are we redesigning roles and tasks while ensuring quality?
Where do agents sit within the organization, and who is accountable for their performance?
These questions are about when to rely on AI. Mostly, though, they are questions about when to hold back.
And that still matters more.
Quotes were lightly edited with an LLM for confidentiality and clarity. We apply our proprietary methodology alongside AI-augmented analysis in our own work.
*********
