Agentic AI risk management: control what the agent can do, not what it says

The short version

• The main risk in agentic AI is the actions an agent can take, not the text it produces.
• An AI agent should run on its own credentials, with only the permissions its task needs.
• Irreversible actions, such as payments, deletions and external messages, should wait for human approval.
• Assume an agent will sometimes follow instructions hidden in content it reads, and limit what that can achieve.
• Autonomy should widen in stages, and only where measured results on real tasks support it.

What is agentic AI risk management?

Agentic AI risk management is the work of controlling what an AI agent is allowed to do, how far a single mistake can spread, and how quickly someone finds out. It is about actions more than words. An agent reads, decides and then calls tools that change things in real systems.

That shift is the whole point. A chatbot that gets something wrong produces a bad paragraph, and a person decides what to do with it. An agent that gets something wrong has already sent the email, issued the refund or edited the record. The controls that matter are the ones that sit between the model's decision and the system it acts on.

This is why most of the useful work is ordinary engineering. Permissions, approval steps, logs, limits and an off switch. None of it depends on the model behaving well, which is the one property you can least guarantee.

Why do agentic AI projects get cancelled?

Mostly because the cost, the value or the risk was not understood before the build started. Gartner forecasts that over 40% of agentic AI projects will be cancelled by the end of 2027, and names escalating costs, unclear business value and inadequate risk controls as the reasons. That is an analyst forecast, not survey data, but the three causes are ones engineers recognise.

Risk controls are the cheapest of the three to fix early and the most expensive to fix late. Adding an approval step to a design is an afternoon. Adding one after the agent is live means changing a process people have already built habits around, usually after something has gone wrong.

What are the main risks of an AI agent?

The main risks are excessive permissions, instructions hidden in content the agent reads, irreversible actions, runaway loops, silent errors and data leakage. Each one has a specific control, and none of the controls rely on the model choosing correctly.

Silent errors deserve a word, because demos never show them. A wrong action that looks plausible does not raise an alarm. It sits in the CRM or the ledger until someone downstream trips over it, and by then the trail is cold unless every step was logged.

Which control addresses each agentic AI risk?

How should you scope an AI agent's permissions?

Give the agent its own identity and only the permissions its task needs, starting with read-only access. Never let it run on a human administrator's credentials. If the agent's token leaks or the agent is manipulated, the damage is then limited to what that token can do, not to what the most senior user can do.

Access control is a recurring gap where AI breaches occur. In a Ponemon Institute study sponsored by IBM, covering breaches from March 2024 to February 2025, 13% of organisations reported a breach of an AI model or application, and 97% of those said they lacked proper AI access controls. The 97% applies only to the organisations that reported a breach, not to everyone, but it shows what those organisations had in common.

In practice, scoping means separate credentials per agent, write access granted per tool rather than per system, and a review whenever a new tool is added. An agent's permissions tend to grow one convenient exception at a time. Treat each addition as a change that needs a reason.

Which agent actions should need human approval?

Any action that is irreversible, leaves the organisation, moves money or changes who can access what should wait for a person to approve it. Everything else can usually run without a gate, provided it is logged and can be undone.

Approval only works if the person approving can see what will happen. 'Send email' is not enough. The approval screen should show the recipient, the full text and the reason the agent chose to send it. A person shown a raw tool call in JSON will approve it, because they cannot tell what it does.

Gate too much and the gate stops working. If every step needs a click, people click yes without reading, and you carry the cost of a human in the loop without the protection. Pick the few actions where a mistake is expensive and gate those properly.

How do you protect an AI agent from prompt injection?

You cannot fully prevent it, so you limit what a successful injection can achieve. Assume that any email, web page, ticket or document the agent reads might contain instructions, and that the agent will sometimes follow them. Then make sure the agent reading that content cannot take a damaging action on its own.

The practical pattern is separation. An agent that summarises inbound email should not also hold the tool that issues refunds. If the reading step and the acting step must be connected, pass structured data between them, such as an order number and a category, rather than free text that can carry instructions. Restrict outbound destinations to an allow-list, so a manipulated agent cannot send data to an address it found in a message.

What should you log for an AI agent?

Log every tool call with its inputs, its outputs, the request that triggered it, and the model and prompt version in use. That is the minimum needed to answer the question that follows every incident: 'what did it do, and why?'

Logging model output alone is not enough, because the action happens in the tool call, not in the text. Keep logs long enough to investigate a complaint that arrives weeks later, and no longer than your data obligations allow. Logs from an agent that handles personal data are themselves personal data.

How do you stop an AI agent that is going wrong?

Build the stop before you need it: a switch that disables the agent's tools immediately, plus hard limits on steps, spend and run time. Then test the switch, because an off switch nobody has tried is a hope, not a control.

Limits catch the failures that do not look like failures. An agent stuck retrying a failing call will keep spending until something stops it. A step limit, a daily spend cap and a timeout per task turn an expensive surprise into an alert. When a limit trips, the agent should stop and hand the task to a person, not look for a cleverer route around it.

How do you decide when an AI agent can act on its own?

Widen autonomy in stages, and only on evidence. Run the agent in shadow mode first, where it proposes actions and a person takes them. Then let it act with approval on everything. Then remove approval for the low-risk actions where its record is clean. Each stage should rest on measured results against real tasks, including the awkward ones.

Few organisations are at the fully autonomous end. A Gartner survey of IT application leaders at organisations with at least 250 full-time employees found just 15% were considering, piloting or deploying fully autonomous AI agents. Smaller businesses have less margin for an agent's mistakes, not more, so starting with approval on everything is a reasonable default.

When is an AI agent not worth the risk?

When the steps are known in advance. If a process always runs in the same order, a fixed workflow with a model call at one or two points does the job with far less risk, because it can only do what it was written to do. An agent earns its place when the path genuinely varies from one case to the next.

It is also not worth it when nobody will own it after launch. Every control here assumes someone reads the alerts, reviews the samples and updates the permissions. An agent with no owner is a risk that grows quietly, and cancelling it early is a better outcome than finding that out from a customer.