Agentic software engineering: what it is and what to check before you buy

The short version

• Agentic software engineering means an AI agent plans, edits, tests and revises code over many steps, instead of suggesting one line at a time.
• An agentic software engineer proposes changes, and a person stays responsible for every one that ships.
• The productivity gain is unsettled: METR, which ran a randomised trial of AI-assisted development, warns that selection effects make its own signal unreliable.
• AI-generated code still needs security review: Veracode's October 2026 testing of code from 18 AI models found an average pass rate of 62%.
• Coding agents work best on bounded tasks that a test, type checker or build can verify, and worst where nothing checks the result.

What is agentic software engineering?

Agentic software engineering is handing a software task to an AI agent that works through the steps itself: it reads the code, makes a plan, edits files, runs the tests, reads the failures and tries again. It finishes by proposing a change, usually a branch or pull request, for a person to review.

What separates it from earlier AI coding tools is the loop, not the model. Autocomplete predicts the next few lines. A chat assistant answers a question and stops. An agent is given tools, such as a terminal, a test runner and version control, and keeps acting until it judges the task done or runs out of attempts.

That loop is the source of both the value and the risk. An agent that can run commands can fix a failing build without being asked twice. For the same reason, it can delete a file, install a package or rewrite a test.

How does a coding agent differ from other AI coding tools?

What is an agentic software engineer?

An agentic software engineer is a coding agent sold as a team member rather than a tool. It picks up tickets, writes code and opens pull requests. It is still a tool. It proposes changes, and a person stays responsible for every one that ships.

The label invites the wrong mental model. A human engineer carries context the agent lacks: why a module was written the odd way it was, which customer breaks if a field is renamed, which test is flaky and which one guards something important. An agent knows what is in the repository and what you tell it. Anything that lives only in people's heads is invisible to it.

It also carries no consequences. It will not attend the incident review. That is not a reason to avoid it. It is a reason to decide, before you start, who reviews its work and what that reviewer is accountable for.

What does the evidence say about agent productivity?

The evidence does not support a settled productivity figure. One of the more rigorous attempts to measure it, a randomised controlled trial by METR, covered 57 developers across 143 repositories and more than 800 tasks. METR itself warns that selection effects make the signal unreliable, and has announced it is changing its experiment design.

If the people running a randomised trial say their own result cannot be relied on, a vendor's headline percentage deserves more suspicion, not less. Benchmarks measure tasks chosen because they are measurable. Your backlog was not chosen that way.

Google's DORA research programme surveyed nearly 5,000 technology professionals for its 2025 report on AI-assisted software development. Its framing is that AI acts as an amplifier, magnifying what a team already does well and what it already does badly. That fits the mechanism. An agent in a codebase with good tests and clear ownership has something to check itself against. An agent in a codebase without them does not.

Surveys and usage data answer narrower questions. More than 30,000 people took Stack Overflow's 2026 Developer Survey, which reports some results specifically among respondents who use coding agents. Anthropic analysed 500,000 coding-related interactions on Claude.ai and Claude Code. Both tell you what developers do and believe. Neither tells you whether the code was right, and Anthropic's figures are the vendor's own usage data.

What can each source tell you, and what can it not?

Is code written by AI agents secure?

AI-generated code is not secure enough to ship unchecked. Veracode's lab testing in October 2026 put code from 18 AI models, in four languages, through checks for four CWE vulnerability categories. The average pass rate was 62%, which Veracode described as the first meaningful gain since 2025.

Read that the right way round. It is an improvement, and an average pass rate of 62% still means failing more than a third of the time. Treat agent output the way you would treat a pull request from a capable stranger: welcome, and reviewed. Keep static analysis and dependency scanning in the pipeline, because they do not get tired on the fortieth pull request of the week.

Agents add a risk that code suggestion tools did not have: access. An agent runs commands with whatever permissions it was given. If it can read production credentials, an instruction planted in a dependency's documentation or an issue comment can try to make it use them. Run agents in a sandbox, give them only the access the task needs, and keep secrets out of reach.

Where does agentic software engineering work well?

Agentic software engineering works best on bounded tasks with an objective check. When a test, a type checker or a build can say whether a change is right, the agent can iterate against that check without a person approving every step.

• •Upgrading a dependency and fixing whatever the upgrade breaks.
• •Fixing a bug that already has a failing test or a clear reproduction.
• •Mechanical changes across many files, such as renaming an internal API.
• •Writing tests for existing code, provided a person reads what each test actually asserts.
• •First drafts of small, well-specified features in a codebase with established patterns.

When is agentic software engineering not worth it?

Agentic software engineering is not worth it where nothing checks the result, or where the hard part is deciding what to build. An agent is fast at producing code. It is poor at noticing that the requirement was wrong.

• •Codebases with few tests, where a wrong change looks the same as a right one.
• •Work where the specification emerges from conversations with users.
• •Architectural decisions whose costs show up months later.
• •Authentication, payments and permission logic, unless review is senior and unhurried.
• •Teams whose reviewers are already the bottleneck. More pull requests make that worse, not better.

What should a buyer check before adopting a coding agent?

Check your own readiness before comparing products. Whether an agent helps depends mostly on your tests, your review capacity and what the agent can reach, and those answers are the same whichever tool you choose.

1. Tests: would your suite catch a plausible wrong change? If not, fix that first. It pays off with or without an agent.
2. Sandbox: where does the agent run, what can it read, and can it reach the network, production data or credentials?
3. Review: who reviews agent pull requests, and is that time budgeted or taken from work they were already doing?
4. Measurement: run it on a sample of your own tickets and compare with how similar tickets actually went.
5. Cost: count review time and rework, not only the subscription or the token bill.
6. Data: what does the vendor keep of your code and prompts, for how long, and is any of it used for training?
7. Exit: if you stopped tomorrow, would anything be left in the codebase that only the agent understood?

Should you build your own coding agent?

Most teams should not build their own coding agent. General coding agents are a crowded product category, and a custom version built by a small team will struggle to keep pace with vendors who work on nothing else. Spend the effort on what surrounds the agent: tests, sandboxing, review rules and measurement.

Building your own agent makes sense when the task is specific to your organisation and no product covers it, such as triaging support tickets against your own runbooks or reconciling records between two internal systems. The discipline is the same as for coding agents: narrow tools, an objective check at each step, permissions scoped to the task, and a person who signs off on anything that cannot be undone.

Open Dimension AI has no case studies or client results to cite on any of this, and says so. Treat every vendor's productivity figure, including any we publish in future, as a claim to test on your own work.