Agentic system architecture: a buyer's guide to five decisions

The short version

• An agentic system is a model in a loop with tools, so the architecture is the set of constraints on that loop.
• Start with one agent and a small tool set, because one transcript is far easier to debug than five.
• Multi-agent designs buy parallel breadth and pay for it in tokens, so the win has to be breadth-shaped.
• Irreversible actions belong in a separate class, gated by a check the model cannot edit or skip.
• If you cannot replay what an agent did and why, you have a demo rather than an architecture.

What does agentic system architecture actually decide?

It decides what the system is allowed to do without asking you first. Everything else follows from that: the model, the framework, the diagram with boxes and arrows. An agentic system is a model in a loop with tools. It reads a goal, chooses an action, observes the result, and chooses again. The architecture is the set of constraints on that loop.

This is why agent projects fail in ways ordinary software does not. A conventional service does what the code says. An agent does what the loop permits, and the loop permits more than anyone listed, because the model generalises from the tools it has been handed. If a tool can delete a record, the agent can delete a record, in a situation nobody wrote a test for.

So a buyer's guide to this subject is really a guide to four questions. What may the agent call? What may it remember? Who decides when it stops? How do you find out afterwards what it did? Most pitches answer the first question well and treat the other three as implementation detail. The other three are what you live with.

Do you need multiple agents, or one agent with more tools?

Start with one. A single agent with a small, well-chosen tool set is cheaper to run, easier to evaluate and much easier to debug, because there is one transcript to read. Add agents when the work genuinely splits, either into searches that can run at the same time or into skills that need different tools and different instructions. Do not add them because the diagram looks more serious with them in it.

Anthropic's write-up of its own production research system is worth reading because it reports both sides of the trade. In their internal research evaluation, a lead agent delegating to subagents beat a single-agent configuration by 90.2%, which is not a rounding error. The same report measures the bill: agents use roughly four times the tokens of a chat interaction, and multi-agent systems roughly fifteen times.

The useful reading of that pair is about shape, not scale. Multi-agent helps when the task is breadth-first, where several independent lines of enquiry can run at once and be compressed into one answer. It helps much less when the steps depend on each other, because a subagent cannot see what its sibling learned halfway through, and the lead agent ends up re-explaining context it has already paid for. If your problem is a sequence, one agent with a good tool set will usually beat a committee.

Which shape fits which problem?

Where should state live in an agentic system?

Outside the model, in something you can query. The context window is working memory for one run, and it is the wrong place to keep anything you will need tomorrow. Treat it as a scratchpad that gets thrown away, and put durable facts in a store with a schema: a database row, a document with a known shape, a file with a version on it.

'We keep everything' is an awkward answer to a later request, not a memory design. Once an agent writes freely into shared memory you own every problem of a database with no migrations: stale facts that outlive their truth, two entries that disagree, and no way to say which one the agent used on the day a customer complained. Decide what is allowed to persist, for how long, and who can read it, before the first write happens.

Retrieval is part of the same decision and carries its own rule. Resolve who is asking before you search, not after. Filtering results afterwards still means the model read content the user cannot open, and a model that has read something can be argued into repeating it. In an agentic platform architecture this gets worse than in plain search, because the agent can search many times per run, down paths nobody anticipated.

How do you stop an agent doing something irreversible?

By making irreversible actions a different class of thing, with their own gate. Reads and drafts can be cheap and free. Anything that moves money, sends a message outside the company, deletes data or changes a record a human relies on should be a proposal the agent produces and something else approves: a person, or a narrow piece of code that checks the proposal against rules the model cannot edit.

The practical version of this is dull and effective. Write actions get idempotency keys so a retry cannot double-charge. Tool calls get allowlists rather than free-form endpoints. Every run gets a budget in tokens, tool calls and wall-clock time, and dies when it hits the budget. Every outbound action gets logged with the inputs that produced it. None of that is interesting, and all of it is the difference between an incident you can explain and one you cannot.

The failure to design for is not an agent going rogue. It is an agent doing exactly what it was asked, twice, during a retry storm, at 2am, with nobody watching the queue.

What does an agentic platform architecture cost to run?

More than the equivalent chat feature, and the multiplier is the number to plan with. Anthropic measured agents at about four times the tokens of a chat interaction, and multi-agent systems at about fifteen times. That is per task, so the budget question is not 'what does one request cost' but how many tasks a day, times how many tool calls per task, times how many retries you are willing to tolerate.

Two things usually dominate the bill, and neither is the model you picked. The first is loops: an agent that cannot tell it has failed will try again, and a loop with no budget is an open invoice. The second is context re-sending, because every step of a long run re-reads the history, so transcript length gets multiplied by step count. Trimming what goes back into the loop is often the largest single saving available.

The bigger cost is still people. Somebody has to write the evaluations, read the transcripts when a run goes wrong, and keep tool integrations working when the systems underneath them change their response shapes. Engineering time, not inference, is what makes these systems expensive to own. A plan that budgets for tokens and not for that is incomplete, and it will be the part that slips.

How do you know it works before it ships?

You build a set of tasks with known answers and run the whole system against them, repeatedly, before any user sees it. Unit tests on individual tools are necessary and not sufficient, because the thing under test is a choice, and choices vary between runs. Run the same task several times and look at the spread, not at the best result you got.

Grade two separate things. Outcome: did the system end in the right state? Trajectory: did it get there in a way you would be willing to pay for and defend? A run that reached the right answer after nineteen tool calls and one lucky guess is a failure waiting for a slightly different input. Outcome-only grading hides exactly the fragility you need to see.

Then keep grading after launch. Sample real runs, read them, and write the surprising ones back into the evaluation set. Agentic systems degrade quietly: a tool changes its response shape, a document store fills up with new phrasing, a model version shifts its preferences, and nothing throws an error anywhere. The only early warning is a graded sample that somebody looks at every week.

Is agentic systems architecture different for a small team?

The decisions are the same. The budget for mistakes is smaller, so the discipline matters more. A small team cannot absorb a week of debugging a five-agent system, which is a good argument for starting with one agent, a handful of tools, and a person approving anything that leaves the building.

It is also not an enterprise-only subject. LangChain's 2025 practitioner survey drew 1,340 responses, 49% of them from companies with fewer than 100 people. Salesforce's 2026 survey of 2,025 agentic AI decision makers across 20 countries included 21% small and medium businesses of 21 to 300 employees. Whatever the state of the practice is, small companies are inside it rather than watching from outside.

What changes with size is not the architecture but what you can safely own. The honest version for a small team: pick the narrowest task that has a clear right answer, give the agent the fewest tools that can finish it, and keep a human on the last step until the evaluation set says you can take them off.

What to ask whoever is building this for you

• •Which actions can this system take without a human approving them, and what happens to those actions on a retry?
• •Where does state live, how long does it live, and who is allowed to read it?
• •Show me the transcript of a run that failed, and tell me how you noticed it had failed.
• •What is the per-task token cost at the volume we expect, and what happens to it when the task gets harder?
• •What is in the evaluation set, who wrote it, and how often does it run?
• •If we change the model underneath this next year, what breaks?