Skip to content

Agent Orchestration: Workflows, Agents and Everything in Between

Every team that builds an AI agent eventually has to decide who is in charge of the flow: the code or the model. Between a hard-wired sequence of prompts and an agent that picks its own tools there is a whole range of control patterns. This guide sorts them by level of autonomy, explains the building blocks and offers a way to choose for common business processes such as quoting, support tickets or inspection reports.

What is agent orchestration?

Agent orchestration is the control logic that decides in which order model calls, tools, checks and human approvals happen. It determines whether a process behaves as predictably as a form or whether the model plans its own route. In a company, it is the layer that turns a chat prototype into a dependable process.

In its guide Building effective agents, Anthropic draws a useful line. Workflows are systems where models and tools are orchestrated through predefined code paths. Agents are systems where the model dynamically directs its own process and tool use. People use both words loosely, but the distinction matters for architecture because it shapes how you test, monitor and take responsibility for the system.

Orchestration is needed wherever one model call is not enough:

  • Document processes: read a request for quotation, extract line items, check them against the product catalogue, draft a quote.
  • Support processes: classify a ticket, find the relevant manual sections, propose a reply, hand over to a person when confidence is low.
  • Research and analysis: search several sources, merge findings, write a report.
  • Long-running processes: anything that waits for an approval, a supplier reply or a goods receipt.

What levels of autonomy are there?

Think of the patterns as a scale. At one end the code fixes every step; at the other the model decides what happens next in a loop. Moving along the scale buys flexibility but makes cost, latency and failure modes harder to predict.

LevelWho decides the next step?Typical pattern
1 Fixed chainthe code, always the same wayprompt chain
2 Branching workflowthe code, based on a classificationrouting, parallelisation
3 Planned runthe model plans once, the code executesplanner-executor, orchestrator-workers
4 Free loopthe model, after every stepReAct agent

Two cross-cutting concerns apply at every level: what triggers or wakes the flow (event-driven design), and how its state survives hours or days (long-running execution). We cover both separately below, because in real business processes they often matter more than the choice between a chain and a loop.

How do the control patterns work?

Each pattern gives a different answer to the question "what happens after this step?". Chains and routing settle it in code, planner-executor lets the model plan once up front, and ReAct lets the model decide again after every step. Here is how each one is built and where it fits.

Chain (prompt chaining)

The task is split into fixed sub-steps, each consuming the output of the previous one. You can add programmatic checks in between, such as "does the extraction contain a customer number?". Anthropic recommends this pattern when a task decomposes cleanly into fixed subtasks, trading some latency for higher accuracy at each step.

Example: take a delivery note PDF, extract the line items as JSON, compare them with the purchase order and describe any discrepancies. Every step has a clear input and output format and can be tested on its own.

Routing

A first step classifies the input and sends it to a specialised follow-up. The classifier can be a language model or a conventional classification model. According to Anthropic, routing works well when there are distinct categories that are better handled separately and the classification itself is reliable. One example given there is sending easy questions to smaller, cheaper models.

In customer service this might mean complaints, spare-part requests and invoice queries each go to their own chain, with their own prompt, tools and approval rules.

Parallelisation

Several model calls run at the same time. Anthropic names two variants. Sectioning splits a task into independent parts, for example one call answers the user while another screens the request for disallowed content. Voting runs the same task several times and compares the outputs, for example multiple review passes over the same piece of code.

Planner-executor and orchestrator-workers

Now the model plans. In a planner-executor setup, one call first writes a plan made of subtasks, then the subtasks are executed. Research describes the idea as Plan-and-Solve prompting (Wang et al., ACL 2023): devise a plan that breaks the task into subtasks, then carry them out.

The orchestrator-workers pattern from the Anthropic guide goes further. A central model breaks the task down dynamically, delegates subtasks to workers and synthesises the results. Unlike plain parallelisation, the subtasks are not fixed in advance; they follow from the specific input. When this turns into a full system of agents with separate roles, and what that costs, is covered in the guide to multi-agent systems.

Evaluator-optimizer

One call produces a result, a second call assesses it and gives feedback, and the loop repeats until the result passes or a round limit is reached. Anthropic suggests it where evaluation criteria are clear and iterative refinement demonstrably helps. In business settings that often means text checked against a written checklist.

Free loop (ReAct)

The classic agent: the model reasons about the next step, calls a tool, reads the result and decides again. The pattern comes from the paper ReAct: Synergizing Reasoning and Acting in Language Models by Yao et al. (2022), which has models interleave reasoning steps with actions. The number of steps is not known in advance; you only bound it through stop conditions and budgets.

Which pattern should you choose?

Pick the lowest level of autonomy that solves the task reliably. Anthropic's own principle is to find the simplest solution possible and add complexity only when it demonstrably improves outcomes. A free loop is justified only when the sequence of steps cannot be fixed beforehand.

A short decision path:

  1. Are the steps the same for every case? Use a chain.
  2. Are there a few clearly separate case types? Put routing in front of several chains.
  3. Are some subtasks independent of each other? Parallelise inside the chain.
  4. Do the subtasks depend on the input but stay stable once planned? Use planner-executor or orchestrator-workers.
  5. Does each next step only become clear from the previous result? Use a ReAct loop, with a step limit, scoped tool permissions and approvals.

In practice most business processes end up as a hybrid: a fixed backbone of routing and chains, with a few steps running as a bounded agent loop. A quoting process is a good illustration. The frame is fixed (read the request, produce the quote, approve, send). Only the search for matching products and earlier quotes is open-ended enough to hand to an agent with search tools. The pages on quotation agents and technical service agents show what that looks like in concrete processes.

A warning sign: a team chooses a free loop "because agents are more flexible", yet could describe the whole process in five sentences. That process is a workflow.

What about event-driven and long-running flows?

Most business processes are not started from a chat window. They start from events: an incoming email, a new record in the ERP, a schedule. And many do not finish in seconds; they wait for people or external systems. These two properties shape the architecture more than the choice of control pattern.

Event-driven flows

An event-driven agent is started or resumed by a trigger such as a webhook, a queue message, a new file or a timer. Three things matter:

  • Idempotency: if the same event arrives twice, the quote must not go out twice. Every run needs a unique key.
  • A clear input contract: define exactly what the event contains. The more structured the trigger, the less the model has to guess.
  • A return path: results and errors have to land somewhere. A background agent needs logging and alerting.

Long-running flows

A long-running flow lives for hours, days or weeks, for instance a quote waiting for the sales manager's sign-off. Its state must be stored outside the running process. A simple in-memory agent loses everything when the server restarts.

Two common approaches:

  • Agent frameworks with persistence: LangGraph stores a run's state as checkpoints per thread. Its documentation notes that the in-memory checkpointer loses all checkpoints when the process restarts and recommends a durable store such as PostgresSaver for production. Calling interrupt() pauses a node until the graph is resumed with Command(resume=...) and the same thread ID. The interrupts documentation points out that the node restarts from the beginning on resume, so any code before interrupt() runs again.
  • General-purpose workflow engines: Temporal describes its workflows as resilient and, according to its documentation, able to keep running for years even if the underlying infrastructure fails. After a crash it rebuilds state by replaying the recorded event history. Model calls can then be wrapped as individual activities inside the workflow.

Approvals in long-running flows raise the same questions as any human-in-the-loop step: who may approve, what do they see, and what happens if nobody answers? The guide to human-in-the-loop AI agents goes into detail.

What do you need before you start?

Before picking a pattern you need a documented process, clean interfaces to the systems involved and a way to check results. If any of these is missing, every level of autonomy becomes unreliable, and the free loop most visibly, because it papers over gaps by guessing.

Process: write down how an experienced employee handles the task today, exceptions included. That description often shows which parts are fixed and which are open.

Systems and tools: every tool the agent uses needs a clear description, unambiguous parameters and readable error messages. Anthropic lists careful design of this agent-computer interface as one of its three core principles. Standard connectors such as the Model Context Protocol (MCP) help here.

Data: if a step draws on company knowledge, that knowledge has to be findable and current. A chain built on a poor knowledge base produces poor results very consistently.

Testability: collect test cases with expected outcomes before you compare patterns. Only then can you see whether more autonomy actually helps. These evaluation sets (evals) become the yardstick for every later change to the flow.

Ownership: decide who owns the flow, who grants approvals and who gets alerted when it fails.

How do you build it, step by step?

The reliable route starts with the simplest pattern and adds autonomy only where measurements justify it. That keeps the system explainable, and every extra freedom given to the model has a reason. The steps below apply whether you use a framework or code directly against a model API.

  1. Break the process down: steps, decisions, exceptions, systems involved and approvals.
  2. Fix what is fixed: anything that always runs the same way becomes code or a chain.
  3. Mark the open parts: only steps whose path depends on the input are candidates for planning or a loop.
  4. Build and document tools: read-only tools first; tools that write data only with an approval step.
  5. Collect test cases: real, anonymised cases with the result a person would expect.
  6. Build the simplest version and measure it: start with a chain or routing, then see where it fails.
  7. Add autonomy deliberately: introduce a bounded loop only where the simple version demonstrably falls short.
  8. Harden operations: step limits, cost budgets, a log entry for every tool call, alerts on aborted runs.

On frameworks, Anthropic advises starting with the model API directly, since many patterns take only a few lines of code. Frameworks can hide prompts and responses behind abstractions and make debugging harder; if you use one, understand the code underneath. They are most useful where you would otherwise have to build persistence, pauses and tracing yourself.

What are the limits and risks?

Each level fails in its own way. Fixed workflows break on cases nobody anticipated. Free loops can spin endlessly, take expensive detours and let small errors compound over many steps. Anthropic explicitly warns about the higher cost and the risk of compounding errors that come with autonomous agents.

Common problems in practice:

  • Too much autonomy too early: an agent is asked to "handle the process" although nobody has written the process down. The result is behaviour that is hard to reproduce.
  • No stop conditions: a loop without step and cost limits can, at worst, keep going until someone notices the bill.
  • Hidden state: the flow depends on conversation history held in memory; one restart and it is gone.
  • Duplicate execution: a redelivered event or a re-run node performs a write action twice.
  • Write tools without approval: emails, bookings and orders should be confirmed by a person in early stages.
  • Framework lock-in: the more logic lives inside one framework, the more a later switch costs.

When the approach does not fit at all: if a process is fully rule-based and has no unstructured input, classic automation without a language model is usually simpler, cheaper and more accurate.

How do you get started?

Start with one well-documented process. Build it as a fixed chain with routing and an approval step, then test it on real cases to see where it breaks. Only at those points add planning or a bounded agent loop. That way autonomy grows with your confidence in the system.

A good first candidate has three traits: unstructured input (emails, PDFs), a result that is easy to check (a draft quote, a ticket reply) and a person who signs off anyway. In our own operations, building the flow first as a plain chain with one approval step, and adding open-ended research steps as a bounded loop later, has worked well.

Further reading: when several specialised agents should work together is covered in the guide to multi-agent systems; how to design approvals in the guide to human-in-the-loop. For concrete use cases of AI agents in business, see the overview of specialist agents.

As of November 2026. The pattern names follow Anthropic's December 2024 guide; framework features refer to the LangGraph and Temporal documentation published at the time of checking.

Frequently asked questions

Is an autonomous agent always better than a fixed workflow?

No. For predictable processes a fixed workflow is usually easier to test, cheaper to run and simpler to audit. Autonomy pays off only when the sequence of steps genuinely cannot be known in advance.

Do I need an orchestration framework?

Not necessarily. Chains, routing and simple loops take a few lines of code against a model API. Frameworks earn their place when you need persisted state, pauses for human input or runs that last for days.

How is a long-running agent different from a normal agent call?

It waits for events such as an approval or a supplier reply, sometimes for days. Its state therefore has to live in durable storage so that it can pick up where it stopped after a restart.

Can I move from a workflow to an agent later?

Yes, and that is the usual path. Replace a single fixed step with a bounded agent loop once your evaluations show the rigid version keeps failing at that point.

Sources

  1. Anthropic: Building effective agents (December 2024) anthropic.com
  2. Yao et al.: ReAct: Synergizing Reasoning and Acting in Language Models (arXiv 2210.03629) arxiv.org
  3. Wang et al.: Plan-and-Solve Prompting (arXiv 2305.04091, ACL 2023) arxiv.org
  4. LangGraph documentation: Interrupts docs.langchain.com
  5. LangGraph documentation: Persistence and checkpointers docs.langchain.com
  6. Temporal documentation: Workflows docs.temporal.io