Agentic analytics: what it does, and what breaks first

The short version

• Agentic analytics is the agent choosing the steps, not just translating one question into one query.
• The failure mode is not a crash. It is a confident wrong number that gets pasted into a board deck.
• An agent cannot resolve a metric definition the company has not agreed on, so it will pick one and sound certain.
• Permissions have to be enforced by the warehouse, not requested in the prompt.
• Routine reporting should stay deterministic. Agents earn their keep on the long tail of questions nobody will build a dashboard for.

What is agentic analytics, and how is it different from asking a chatbot for a number?

It is analytics where the agent chooses the steps. A business question goes in, and the system decides which tables are relevant, writes the queries, runs them against the warehouse, looks at what came back, and decides whether that answers the question or whether it needs another query. What comes out is an answer with the working attached, not a chart you are left to interpret.

Natural language to SQL does something narrower. It turns one sentence into one query and returns the result. If the sentence was ambiguous, the query is wrong, and you find out when a human reads the output and frowns. The agentic version adds a loop: it can notice an empty result, a join that dropped half the rows, a column that is null for most of the period, and try something else. That loop is the whole value of the approach. It is also the whole risk, because a system that retries until the output looks reasonable will eventually produce something that looks reasonable and is not.

A dashboard is different again. A dashboard answers a question somebody anticipated, in a fixed way, and its main failure is that it goes quietly stale or nobody opens it. An agent answers questions nobody anticipated, which is exactly why there is no reviewed definition waiting for it to use.

What does the loop actually look like?

1. Resolve the question against written metric definitions, or refuse if no definition exists.
2. Choose tables from a curated set, not from the whole warehouse.
3. Write SQL and run it under the asking user's own permissions.
4. Inspect what came back: row counts, nulls, date ranges, anything that looks like a failed join.
5. Iterate or stop, under a hard limit on attempts and on bytes scanned.
6. Return the number, the query that produced it, and the definition it used.

Why do metric definitions decide whether this works?

Because the agent has to pick one, and most companies have several. 'Revenue' can mean bookings, billings, recognised revenue or gross merchandise value, depending on whether you asked finance, sales or the product team. 'Active customer' usually has a window attached, and the window differs by department. These are not edge cases. They are the substance of most questions worth asking.

A human analyst handles this by asking, or by knowing which definition the person in front of them meant. An agent will not ask unless it was built to ask, and the default behaviour of a language model is to produce a plausible answer rather than to stop. So it selects a definition, writes correct SQL for that definition, and returns a number that is internally consistent and answers a question nobody asked.

The fix is boring and it is not an AI problem. Write the definitions down somewhere the agent reads them, as a semantic layer or a set of certified metrics, and have the agent refuse questions that touch an undefined term. Teams that skip this step get a system that is fluent, fast and subtly wrong, which is the worst of the available outcomes because nobody catches it.

How do you stop the agent inventing the number?

Never let the model produce the figure. The model's job is to write a query. The warehouse's job is to produce the number, and the number that reaches the user should be the one that came back from execution, carried through unchanged. A model asked to summarise a result set will sometimes round, sometimes recompute, and occasionally fill in a value it did not receive. If the figure passes through the model as prose, you have lost the guarantee.

Show the query. Every answer should carry the SQL that produced it and the definition it used, close enough that a sceptical analyst can reproduce the result in under a minute. This is not a transparency gesture. It is the only cheap way to catch a wrong answer, because a wrong number is invisible and a wrong join is obvious.

Then build in refusal. An agent that cannot find a defined metric, or that gets a result failing its own sanity checks, should say it cannot answer. Saying plainly what is not known is a feature. Most systems are tuned the other way, and that is why people stop trusting them in week three.

How do permissions work when the agent writes the query?

The warehouse enforces them, or nothing does. Run the generated query under the asking user's identity, or under a role that mirrors it, so row-level security and column masking apply the way they do for every other client. The agent then has no special reach, and the question of what it may see has the same answer as the question of what the user may see.

What does not work is instructing the model to avoid certain tables. That is a request, not a control, and a request can be argued around by a cleverly phrased question or by content the agent reads along the way. It also fails silently: there is no log entry for a boundary that was never enforced.

One practical consequence. If the agent summarises results into a shared channel or a scheduled report, the audience of that report is now the audience of the underlying data. Decide that deliberately rather than discovering it.

How does this compare with what you already have?

What does the published evidence actually tell you?

Less than the category's marketing implies. There is serious survey work on this, and it is worth knowing what it covers. MIT Technology Review Insights fielded a global survey for Snowflake in July and August 2026, putting the question to 400 chief data and analytics officers, CIOs, CTOs and chief AI officers across eight industries, all at organisations above $500m in annual revenue. dbt Labs surveyed 363 data practitioners and leaders between December 2025 and February 2026. The 2026 Stack Overflow Developer Survey collected 30,903 valid responses from 169 countries, and asked 14,304 of them about trust in AI output.

Those are sample frames, and they tell you who was asked and when. None of them tells you whether an agent pointed at your warehouse returns your revenue number correctly. That is not a gap in the research. It is the nature of the question, which is specific to your schema, your definitions and the questions your people actually ask.

So the number that should decide this is one you generate. Take thirty to fifty real questions, get an analyst to verify the right answer for each, and run the system against them on every change. You want two figures: how often it is right, and how often it declines to answer rather than guessing. Teams consistently underweight the second.

When is agentic analytics not worth doing?

When your definitions are contested. If two teams report different revenue figures today and the disagreement has never been settled, an agent will not settle it. It will industrialise the disagreement and give it a confident voice, at speed, in more places. Fix the definitions first, and you may find the agent matters less than you thought.

When the questions are already answered. Board reporting, regulatory returns and anything with a fixed cadence should stay deterministic: a reviewed query, a scheduled run, the same output every time. Agents belong on the long tail, the one-off questions that never justified a dashboard and currently wait three days for an analyst.

And when you cannot price the benefit honestly. If you want a number for the analyst time involved, use published wage data rather than a vendor's productivity multiplier. The U.S. Bureau of Labor Statistics publishes national wage estimates for data scientists from its Occupational Employment and Wage Statistics programme, with a May 2025 reference period. That is a defensible basis. A claimed tenfold speed-up is not.