What is agentic testing, and when is it worth using?

The short version

• Agentic testing means an AI agent plans, runs and adapts tests towards a stated goal instead of replaying a fixed script.
• The main risk in agentic testing is a false pass, where the agent reports success on behaviour that is actually broken.
• Agentic testing suits journeys that change often and are tedious to script. It suits exact rules, such as tax or permissions, poorly.
• An agent-run test should end in a deterministic check the agent cannot edit, not in the agent's own judgement.
• Published survey evidence on agentic testing mostly comes from testing vendors and large organisations, so it says little about small firms.

What is agentic testing?

Agentic testing is software testing in which an AI agent decides the steps. You give it a goal, such as 'a new customer can buy a subscription', and it explores the application, chooses actions, reacts to what it sees and reports the result.

The difference from ordinary test automation is where the decisions live. A script holds every step and every expected value, written in advance. An agent holds a goal and works out the rest during the run. That makes it more tolerant of change and harder to predict, and both properties come from the same design.

The term is used loosely. It gets applied to anything from a model that suggests test cases to a system that runs whole journeys unsupervised. The useful question is what the tool decides on its own during a run. If the answer is nothing, it is a test generator, which may still be useful, but it is not an agent.

How does agentic testing differ from scripted test automation?

A scripted test does the same thing every run and fails when anything differs. An agentic test can take a different route each run and fails only when it judges that the goal was not met. Scripts fail too easily. Agents can forgive too easily.

That trade-off decides where each belongs. A broken script is loud: somebody sees a red build and looks. A lenient agent is quiet: the build stays green and the fault reaches customers. Loud failures cost time. Quiet failures cost trust in every result the suite has produced.

Scripted tests and agentic tests compared

Where is agentic testing worth using?

Agentic testing is worth using on user journeys that change often, are tedious to script and have a clear outcome. Sign-up, onboarding, search and checkout on a busy front end are typical examples. These are the flows where script upkeep eats the time that testing was meant to save.

It also suits exploration. An agent told to find a way to break a form will try orders of action nobody wrote down. Treat what it finds as leads for a person to confirm, not as a verdict.

It is not worth using where the correct answer is exact. Tax calculations, pricing rules, permissions and anything a regulator will read belong in deterministic tests, where a reviewer can see what was checked and rerun it. It is also the wrong first step if you have no tests at all. An agent needs someone to define correct behaviour, and that work is the same either way, so start with plain scripts.

What goes wrong with agentic testing?

The main failure is the false pass, where the agent reports success on behaviour that is actually broken. It happens because the same model that drives the test also grades it. Shown an error it has not met before, a model can still conclude 'the order went through'.

The second failure comes from self-healing, the feature that repairs a test when the interface changes. That helps when a button has moved. It hurts when the button has gone because of a bug and the agent finds another route to the goal. The test turns green while a real user stays stuck.

The third is variation. Two runs of the same goal can take different paths, so a failure may not reproduce, and a developer without a full record of the run gets a report they cannot act on. The fourth is side effects. An agent with real credentials will click what it finds, including 'delete' and 'send'.

How do you implement agentic testing safely?

Start with one journey, in a test environment, with pass conditions the agent cannot change. Run it alongside your existing tests, not instead of them, and do not trust a green result until the agent has caught bugs you planted yourself.

Two rules carry most of the safety. The pass condition should be a deterministic check on real state, such as a database row or an API response, not the model's reading of a screen. And the agent should find while scripts hold: each confirmed finding becomes a scripted regression test, so the next run does not depend on the model's judgement.

A first rollout, in order

1. Pick one journey that changes often and has a clear outcome.
2. Give the agent a test environment, seeded data and test accounts, with no production credentials.
3. Write the pass condition as a check on state that the agent cannot edit.
4. Set limits: an allow-list of URLs and tools, a step cap and a spending cap per run.
5. Record every run: steps, screenshots, prompt and model version.
6. Plant known bugs in a test build and count how many the agent catches.
7. Require a person to approve any change the agent makes to a test.
8. Turn each confirmed finding into a scripted regression test.

How do you know whether agentic testing is working?

Agentic testing is working if it catches planted bugs, finds real faults your scripts missed and costs less attention than it saves. The number of tests run tells you nothing. Neither does a clean dashboard.

Planted bugs are the honest measure of any tester. Break something on purpose in a test build, across several kinds of fault, and see whether the agent notices. If it misses them, its passes mean nothing, however tidy the reports look.

Track four things per journey: planted bugs caught, false passes, false alarms and the cost of each real defect found, including review time. Compare them with the scripted tests for the same journey. If a fair trial shows the agent finding nothing the scripts did not, stop. That is a useful result.

How much does agentic testing cost?

Agentic testing costs more per run than a script, because each step can call a model as well as using compute. The larger cost is usually people: someone has to write goals, review findings and chase false alarms, and that time does not fall to zero.

Published price lists show what is being sold. Momentic lists its Scale plan at $500 a month, with test steps, hosted runtime and AI features drawing from one dollar balance and no per-user seat charges. QA Wolf prices mobile and desktop runs higher, bills longer tests in 5-minute blocks and has sales quote its managed services.

These are list prices read in November 2026, shown to illustrate pricing models, not to recommend either product. The practical point is that usage moves the bill, so run frequency matters. Estimate runs per day multiplied by average test length before you commit. Running every agentic test on every commit is rarely the right setting. Nightly runs, or runs limited to the journeys a change touches, usually are.

What does the evidence say about agentic testing?

The evidence is weaker than the marketing suggests. The surveys cited here mostly come from companies that sell testing tools, and they mostly sample senior people or large organisations.

Sauce Labs surveyed 200 US executives and 200 US engineering leaders with Wakefield Research in June 2025. SmartBear surveyed 1,436 technology professionals who own testing and quality in Q3 2026, at organisations with more than 500 employees and over $50 million in annual revenue. Both sell testing products.

Leapwork's 2026 report with SD Times measures AI in general rather than agents, so it cannot be set beside the Sauce Labs results. The World Quality Report 2025 asked more than 2,000 senior executives across 22 countries and 10 sectors. Its own headline says enterprise-level scaling remains elusive, and its productivity findings are self-reported.

The closest peer-reviewed work is narrower. Meta engineers evaluated their own TestGen-LLM tool on 86 Kotlin components for Instagram Reels and Stories, published at ACM FSE 2024. That is a tool for improving unit tests at a very large company, not a fully autonomous testing agent.

Small firms are largely missing. Sauce Labs gives no results for them, SmartBear's sample excludes them and the World Quality Report samples senior executives instead. Any claim about what agentic testing will do for a small team is judgement, including the judgement in this post.

Is testing an AI agent the same as agentic testing?

No. Agentic testing uses an agent to test ordinary software. Testing an AI agent means checking a system whose own behaviour varies, and it needs a different method. The phrases get mixed up, so check which one a vendor or article means.

To test an agent, build a set of representative tasks, score the outcomes, check which tools it called and with what arguments, and repeat the runs, because one pass proves little. Test what it should refuse as well as what it should do. That evaluation work decides whether an agent is ready to ship, and it is a separate job from testing your web application.