Workflow automation

Generative AI workflows: what they are and how to build one that holds up

A generative AI workflow is ordinary software with a model in one or two of its steps, and most of the engineering sits around that model.

Open Dimension AI30 July 20267 min read

What are generative AI workflows?

Generative AI workflows are business processes where a language model handles one or more steps, such as drafting, classifying or extracting, inside a fixed sequence. Ordinary code controls the order, checks each output and decides what happens on failure. A person reviews anything the business cannot afford to get wrong.

The short version

A generative AI workflow is a fixed sequence of steps in which a language model does some of the work and ordinary code controls the rest.
A workflow differs from an agent because code chooses the next step in a workflow, while the model chooses it in an agent.
Most of the engineering in a generative AI workflow sits around the model: intake, validation, routing, review and logging.
A generative AI workflow suits work that is frequent, mostly text and cheap to check.
A generative AI workflow is not ready for production until it has been tested on real past cases, including the ones it should refuse.

What is a generative AI workflow?

A generative AI workflow is a fixed sequence of steps in which a language model does part of the work and ordinary code controls the rest. The model might read an email, extract fields from a document or draft a reply. The code decides what runs next, what counts as a valid output and what happens when a step fails.

The word that matters is fixed. The route through the process is written down in advance, as it would be in any other software. The model is one component, called at a known point, with a defined input and a defined output. That is what makes a workflow testable.

Take a supplier invoice. Code receives the file. A model extracts the supplier, amount and due date into a fixed structure. Code compares the amount with the purchase order. If they match, the invoice joins the approval queue. If they do not, a person sees it with the mismatch shown. Only one step uses a model, and that is normal.

How is a generative AI workflow different from an AI agent?

In a workflow, code chooses the next step. In an agent, the model chooses it, usually by picking from a set of tools and deciding when it has finished. That one difference changes what the system can handle, what it costs and how hard it is to test.

An agent can handle cases nobody listed in advance, which is its real value. The price is that the possible paths multiply with every tool, so you cannot list them all, and a single request can run longer and cost more than you planned. A workflow cannot surprise you that way, because every path already exists in the code.

Start with a workflow. Move to an agent only when the steps genuinely cannot be known in advance, and keep its tools narrow when you do. Plenty of systems sold as agents would be cheaper and more reliable as workflows.

Workflow and agent compared

Question	Workflow	Agent
Who picks the next step?	Code	The model
Can you list every path?	Yes	No
What does a run cost?	Predictable within a range	Varies with how long the model runs
How do you test it?	Step by step, then end to end	On whole outcomes, over many runs
What does it fit best?	A known process, repeated often	An open-ended task with no fixed route

What are the stages of a generative AI workflow?

A production generative AI workflow usually has six stages: intake, preparation, the model call, validation, routing and logging. Only one of them is the model. The other five are ordinary engineering, and they decide whether the system can be trusted on a bad day.

The six stages, in order

Intake: a trigger starts the run, such as a new email, form or file, and code records what arrived.
Preparation: code cleans the input and fetches what the model needs, such as a customer record or a policy.
Model call: the model gets a narrow instruction and returns a defined structure, not free text, wherever code reads the result.
Validation: code checks what it can verify, such as required fields, allowed values, totals that add up and quotes that exist in the source.
Routing: valid, low-risk outputs move on, and everything else goes to a person with the reason attached.
Logging: each run keeps its input, prompt version, model version, output and decision, so a bad result can be traced.

Which processes suit a generative AI workflow?

The best candidates are frequent, mostly text and cheap to check. Triage, document extraction, first drafts and summaries fit well, because a person can confirm the output faster than they could produce it. If checking takes as long as doing the work, the workflow saves nothing.

Volume matters because the build and the upkeep are costs you pay regardless. A task that comes up a few times a month rarely repays them. Stability matters too. If nobody can write down how the process runs today, a model will not fix that, and the first job is agreeing the process.

Some work should stay out altogether. Do not put a model where a rule already works. A lookup, a calculation or an if statement is cheaper, faster and right every time. And keep a person in any step where one wrong output costs more than the workflow saves.

How do you build your first generative AI workflow?

Pick one process, measure how it runs today, and build the smallest version with a person reviewing every output. Widen it only when the numbers justify it. A small first build matters because it exposes problems in the data and the process that no planning meeting finds.

The number that decides when to automate is the edit rate: how often a reviewer changes what the model produced. While that rate is high, the review step is doing real work and must stay. When it falls close to nothing for one category of cases, you have evidence to let that category through without a person.

A first build, in order

Choose one frequent process with a named owner.
Record the baseline: volume, time per case and error rate.
Collect real past cases with known correct answers.
Build it with a person reviewing every output.
Track how often reviewers change the output, and why.
Automate only the categories where changes have nearly stopped, and keep sampling them.

Where should people review a generative AI workflow?

People should review wherever a mistake would be expensive, hard to reverse or visible to a customer. In practice that means before anything is sent, paid, filed or published. Low-risk internal steps, such as tagging a ticket, can often run without a person once the numbers support it.

Review has to be real to count. A queue that shows only the finished draft invites people to approve without reading, especially after a long run of clean cases. Show the source beside the output, highlight the fields that failed a check, and make rejecting as quick as accepting. Then sample approved cases to see whether the review is catching anything.

How do you test a generative AI workflow before it goes live?

Test it against real past cases with known correct answers, and rerun that set on every change. Include the awkward inputs: scanned pages, missing fields, two requests in one email, and cases the workflow should refuse. A demo only proves that the easy path worked once.

Score each step as well as the final result. If extraction is wrong, you want to know that directly, not just that the output failed several steps later. Set the pass mark before you run the test, because a threshold chosen after seeing the numbers tends to match the numbers.

Then run it in shadow. The workflow processes live inputs while people carry on as normal, and you compare the two. This shows the failures your test set missed. Repeat the tests when the provider changes a model, because a new version can behave differently on the same prompt.

What does a generative AI workflow cost to run?

A generative AI workflow has two running costs: model usage and the people who keep it working. Usage depends on how many runs you have and how much text goes in and out, so you can estimate it by running a sample of real inputs. Upkeep is easier to miss. Source systems change, models get replaced, and someone has to own the review queue.

Per-seat assistants are priced differently. Microsoft sells the Microsoft 365 Copilot Business add-on per user per month, paid yearly, and it requires an eligible Microsoft 365 Business plan. Some AI features carry extra usage-based charges. The discount on that page needs an annual commitment and applies to the first year only, so budget on the full price.

A seat and a workflow are not substitutes. A seat helps one person while they work. A workflow runs a process whether or not anyone has it open. Decide which one the problem needs before comparing prices.

Does the evidence show that generative AI saves time?

Partly, but the evidence is thinner than it looks, and none of the sources on this page measures a workflow directly. They measure people using AI tools, or firms saying they use AI. Treat any figure as a reason to measure your own process, not as a forecast for it.

The St. Louis Fed's Real-Time Population Survey work shows why the base matters. The time saved is self-reported, and its 1.6% starting point covers all workers, including nonusers. It is not a per-user saving and should not be quoted as one.

A randomised field experiment on Microsoft 365 Copilot covered over 6,000 workers at 56 firms, which is a strong design. All four authors work at Microsoft, the paper appears to be an unreviewed preprint, and a later paper describes the participants as large firms. The result may not transfer to firms under 50 staff.

Adoption data has its own limits. The US Census Bureau reports that firms with at least 20 employees are the biggest AI users, but its survey covers AI in general, not generative AI alone. The Federal Reserve Banks' Small Business Credit Survey drew 6,525 responses from a convenience sample, and the report says it is not random. Its base is firms using AI. Menlo Ventures surveyed 495 US enterprise AI decision-makers, which shows enterprise behaviour only, and Menlo invests in AI vendors.

The number that counts is your own: time per case before and after, including the checking. Record it before you build, because you cannot recover it afterwards.

Questions people ask

Do we need an agent, or is a workflow enough?

Usually a workflow is enough. If you can write the steps down, code should control them and the model should do the parts that need language. Choose an agent only when the route through the task cannot be known in advance. Even then, keep the set of tools small, set a hard limit on steps and spend, and log every action so you can see what it did.

Can we build a generative AI workflow with no-code tools?

For a simple, low-risk process, yes. No-code tools handle triggers, routing and a model call well enough to prove whether the idea has value. They become harder to live with when you need test sets, version control for prompts or careful handling of sensitive data. Treat a no-code build as a way of learning what the process needs, and be ready to rebuild it if it starts to matter.

How long does it take to build one?

That depends less on the model than on the process around it. A narrow workflow with clean inputs and one system to connect to is a small project. Messy documents, several systems and strict approval rules take longer. We would not quote a general figure, because the honest answer comes from looking at your inputs and how cleanly your systems connect.

What happens when the model gets something wrong?

It will, so design for that. Validation catches the errors code can check, such as missing fields, totals that do not add up or quotes that are not in the source. A person catches the rest, at the points where a mistake would cost money or reach a customer. Logging each run means you can trace a bad output and add that case to your test set.

Is it safe to send company data through a generative AI workflow?

It can be, but safety depends on the setup, not the label. Check the provider's terms on retention and training, send only the fields a step needs, and keep access controls on anything the workflow retrieves. Treat text from outside the business as untrusted, because it can carry instructions aimed at the model. Regulated data needs advice on the rules that apply to you before any of it leaves your systems.

How do we know whether it is paying off?

Compare time per case before and after, and include the time people spend checking. Track how often reviewers change the output, how many cases fall out to a person and what each run costs. If checking takes nearly as long as the original work, the workflow is not paying off, and stopping is a sensible result. You need the baseline from before the build, so record it first.