An AI agent in action is mostly the boring parts

The short version

• An agent run is a loop of tool calls, not a single prompt and a single answer.
• The useful view of an agent in action is the trace: the trigger, every tool call, every result, and the decision to stop.
• Most published demos show one happy path, so they tell you nothing about what happens when a tool returns nothing.
• Cost per run follows the number of steps the model takes, which is a design decision rather than a property of the model.
• If you cannot replay a run from its logs, you cannot debug it, price it or defend it.

What actually happens when an AI agent runs?

A loop runs. The agent receives a trigger, assembles whatever context it has been given, and puts one question to the model: given this goal and these tools, what is the next step? The model either calls a tool or produces an answer. If it calls a tool, the result comes back into the context and the same question is asked again. That repeats until the model answers, a stop condition fires, or the budget runs out.

The model choosing the order is the whole difference between an agent and a form that calls an API. In a fixed automation you decide the sequence of steps and anyone can read it in the code. In an agent you decide the tools, the limits and the stopping rules, and the sequence is chosen at runtime. You gain the ability to handle requests you never enumerated. You lose the ability to know in advance exactly what will happen.

Most of what people mean by 'AI agents in action' sits on top of that loop rather than beside it. Memory is context you have chosen to carry between runs. Planning is the model writing its intended steps into the context before it executes any of them. Multi-agent is one loop exposed to another loop as a tool. None of those change the basic shape, which is why the trace of a single run is the thing worth learning to read.

What does a single agent run look like, step by step?

It looks like a short list of calls with a decision between each one. Take a common case. A customer writes to a retailer: 'my order has not arrived, it has been nine days'. The trigger is the inbound message. The goal is a resolution or a clean handover to a person. The tools are an order lookup by customer id, a carrier tracking call, a replacement action with a value limit, and a handover to a human queue.

That is four tools and, in a healthy run, three or four calls. The table below is the same run written out twice: what the trace records, and what tends to break at that exact point. Almost nothing in the right-hand column is about the model being unintelligent. It is about the systems around it.

One run, traced, with the failure that lives at each step

Where do agent runs actually fail?

At the tool boundary, mostly, and not in the model's reasoning. The failures that reach customers are dull ones. A lookup returns nothing and the agent fills the gap with something plausible. A timeout is indistinguishable from an empty result, so the agent proceeds as though it checked. An action runs twice because the retry carried no idempotency key. A permission that is enforced in the web interface turns out not to be enforced in the API the agent was handed.

The second family is the stopping problem. An agent with a goal it cannot reach and a tool that keeps returning almost useful results will keep calling that tool. There is no error in the ordinary sense, just a run that takes forty steps instead of four. Without a hard cap on steps and spend per run, this arrives as a bill at the end of the month rather than as an alert at the time.

The third is the handover, and it is the one teams under-build. Much of the value of an agent is in the requests it refuses to handle. A run that stops early, says clearly what it did and did not establish, and puts the case in a human queue with the order id and the tracking state attached is a good run. A run that produces a confident answer to a question it could not actually resolve is a bad run, even though it looks better in a screenshot.

How do you tell an agent in action from a demo?

Ask for the second-worst input rather than the best one. A demo is a single path chosen because it works. Ask to see a run where a tool returned nothing. Ask for the full trace, including the calls that failed. Ask what the run cost, how many steps it took, and what the latency was at the ninety-fifth percentile rather than the median. If those numbers do not exist for a prototype, they will not exist after launch either.

The same scepticism belongs on the published evidence, which is thinner than the volume of writing about agents suggests. Most of it is either a self-selected survey or a vendor announcing its own product, and both need their method read before their headline. LangChain's State of Agent Engineering collected 1,340 self-selected responses over two weeks, and 49% of those respondents came from companies with fewer than 100 people, so it describes the people who answer agent surveys more than it describes an industry. Intuit's 'up to 12 hours a month' figure is footnoted to a survey it commissioned of its own QuickBooks Online customers who were using the new bank feed features, with the sample size not disclosed. Neither of those is a reason to ignore the number. Both are a reason not to quote it as a benchmark for your own run.

The large official surveys are more careful and correspondingly less specific about agents. The Federal Reserve Bank of Philadelphia's module on artificial intelligence drew 5,248 responses nationally and 679 from Delaware, New Jersey and Pennsylvania, counting any firm with 1 to 499 employees as a small business. The CFO Survey polled 548 financial decision-makers between 11 November and 1 December 2025, with large firms defined as 500 or more employees. Those are useful for reading direction of travel. They cannot tell you whether the thing you are about to build works.

Pricing pages are often more informative than case studies, because they show what a vendor believes one completed run is worth. Intercom's Fin page lists $9.99 for each lead qualification and a minimum of 50 outcomes a month for use with an existing helpdesk. Whatever you build, your own cost per resolved request is competing with numbers of that shape, and you will not know yours until you have recorded it.

What should you be watching while an agent runs?

Five numbers, all of which have to be recorded at the time, because none of them can be reconstructed afterwards from the customer-facing output. A trace that stores only the final message is a receipt, not a log. What you want is every call with its input, its output, its duration and its cost, keyed to a run id you can search.

The point of recording them per run rather than per day is that agent problems are usually concentrated. The median run is fine. The expensive, slow and wrong runs are a small share of traffic with something in common, and you find that something by sorting traces by step count and reading the top twenty.

Five things to record on every run

• •Steps per run: how many tool calls the model made before it stopped.
• •Cost per run, and separately cost per resolved request, since runs that end in a handover still cost money.
• •Latency at the ninety-fifth percentile, because the median hides the runs where a tool was slow and the model retried.
• •Handover rate, split into clean handovers with context attached and abandonments.
• •Error rate per tool, because one flaky integration is indistinguishable from a bad model when seen from outside.

Which tasks are worth giving to an agent, and which are not?

Give an agent work where the path genuinely varies and the cost of being wrong is recoverable. Order status and delivery exceptions, triaging inbound documents into the right queue, first-line support over a known knowledge base, turning an unstructured message into a structured record. These share two properties: many different entry points for the same underlying request, and a cheap undo if the agent gets it wrong.

Do not give an agent a task you can already write down as a fixed sequence. If the steps are the same every time, a scheduled job is cheaper, faster, easier to test and readable by anyone. Wrapping it in an agent puts a model on the critical path of something that was already deterministic, and the first time it fails you will be reading a trace to find out what it did instead of reading twenty lines of code. This is the most common waste in agent projects and it is entirely avoidable.

Work where a mistake is expensive and hard to reverse wants a different split: the agent prepares, a person commits. Payments above a threshold, anything that becomes a legal filing, clinical advice, anything that touches an employment decision. The agent can assemble the case, cite what it read and draft the action. The irreversible step stays behind a human click, and that is not a temporary measure to be removed once confidence grows.

What does an agent need from your systems before it can act?

An API for every action you want it to take, a permission model it inherits rather than reimplements, and an audit trail that records both the agent and the person it acted on behalf of. Most of the effort in a real build sits here, not in the prompt. If an action currently requires a human to click three screens and there is no endpoint behind it, that is the work, and no amount of model capability removes it.

Three specifics save the most pain later. Make every write action idempotent with a key supplied by the run, because retries are normal and double-shipping is not. Separate read tools from write tools so you can give an agent broad read access and a very short list of writes. Put value and scope limits in the code behind the tool rather than in the instructions, because a limit stated in a prompt is a request and a limit enforced in an endpoint is a limit.

Permissions deserve the same treatment. Resolve who the run is acting for at the start, and scope every tool to what that person could do themselves. An agent that holds a service account with wider access than any of its users is a standing problem, and it will not show up in testing, because tests check whether the right answer came back rather than whether the wrong record was reachable.