When an agent works well, the model gets the credit. When it fails, the failure is usually somewhere else — a loop that never terminated, state lost on restart, a tool error swallowed silently, context that overflowed mid-task. Almost none of those are model problems. They are orchestration problems, which is where the real engineering of an agent lives.
What Orchestration Is
Agentic orchestration is everything in an agent that is not the model.
The model contributes one thing: judgement about what should happen next, expressed as text. Orchestration contributes everything required to turn that into a system — control flow, state, execution, bounds, recovery and observability.
A useful way to see the division:
| The model provides | Orchestration provides |
|---|---|
| A judgement per call | When to call, how many times, when to stop |
| A request to use a tool | Validation, authorisation and execution |
| Text output | Parsing, routing and persistence |
| Nothing between calls | All state and continuity |
| No awareness of cost or time | Budget, limits and termination |
Why the Model’s Limits Define the Job
Orchestration’s responsibilities are not arbitrary. Each one exists because of a specific structural property of the model.
| Model property | Orchestration must therefore |
|---|---|
| Stateless — retains nothing between calls | Own all state and rebuild context every request |
| Cannot execute — only emits requests | Validate and perform every action |
| Non-deterministic — same input can vary | Handle variation, retries and unexpected output |
| Finite context | Budget, prioritise and truncate |
| No sense of progress | Detect completion and enforce termination |
| Can be wrong confidently | Verify, guard and provide fallbacks |
Read that table in reverse and it becomes a specification. The orchestration layer is the answer to a model that cannot remember, cannot act, cannot repeat itself exactly, cannot see beyond its window, and cannot tell when it is done.
The Core Responsibilities
Control flow
The loop belongs to the orchestrator. It decides whether to call the model again, whether to execute a requested tool, whether to branch, and when to stop. The model contributes one step at a time and has no view of the loop’s existence.
State management
Conversation history, intermediate results, working variables, tool outputs and task progress all live in orchestration. This is also what makes an agent resumable — if state is durable, a crashed or paused run can continue rather than restart.
Durability separates a demo from a system. An agent holding state only in process memory loses everything on restart, which is tolerable for a chat turn and not for a task running twenty minutes.
Context assembly
Deciding what goes into each call — instructions, tools, retrieved material, history — and what gets cut when the budget binds. Covered in detail in how an agent builds the prompt.
Execution and authorisation
Running tools the model requests, after deciding whether they should run at all. This is a security boundary: a tool request is a suggestion from a text predictor whose input may include untrusted content, so authorisation must be determined independently of why the model asked.
Bounding
Every dimension of an agent run needs a limit, because the model provides none.
| Bound | Prevents |
|---|---|
| Maximum iterations | Infinite loops |
| Token budget | Context overflow and runaway cost |
| Wall-clock timeout | Runs that never return |
| Cost ceiling | Unbounded spend |
| Tool call limits | Repeated calls to a failing dependency |
| Recursion depth | Sub-agents spawning sub-agents indefinitely |
Error handling and recovery
Deciding what a failure means. Some errors should return to the model as data so it can correct itself. Some warrant a silent retry. Some should abort the run. Getting this wrong in either direction is costly — too eager to abort and recoverable mistakes become user-visible failures; too eager to retry and a broken dependency gets hammered.
Observability
Recording what was sent, what came back, which tools ran with what arguments, how long each step took and what it cost. Since the model retains nothing, orchestration is the only place a record can exist. An agent without tracing is not debuggable — you cannot inspect a decision that left no trace.
Human-in-the-loop
Pausing for approval on consequential actions, surfacing a decision to a person, and resuming afterwards. This requires durable state, which is why approval gates are hard to retrofit onto an agent that keeps state in memory.
Orchestration Patterns
| Pattern | Shape | Suits |
|---|---|---|
| Single loop | One agent, one tool set, iterate until done | Most tasks; the sensible default |
| Pipeline | Fixed sequence of steps | Known, stable workflows |
| Router | Classify the request, dispatch to a handler | Distinct request types |
| Supervisor and workers | A coordinator delegates to specialised sub-agents | Separable sub-tasks, large tool sets |
| Parallel fan-out | Independent branches run concurrently, results merged | Independent sub-tasks; latency-sensitive work |
| State graph | Explicit nodes and transitions | Branching or cyclic flows needing checkpoints |
Two notes on choosing. Start with the single loop. Multi-agent architectures are frequently adopted before they are needed, and they multiply failure modes — coordination overhead, context duplication and errors that cascade across agents.
And the strongest practical argument for delegation is not specialisation but context isolation. A sub-agent sees only its own tools and history, which keeps the parent’s context clean and its tool set small — directly addressing the selection degradation described in tool design.
Where Agents Actually Fail
The failure modes that matter in production are almost all orchestration failures.
| Failure | Cause | Prevention |
|---|---|---|
| Runs forever | No iteration cap; model keeps requesting tools | Hard limits on every dimension |
| Repeats a failing call | Error returned without enough information to change approach | Informative errors; detect repetition |
| Loses the thread | Early instructions truncated as context filled | Durable instructions in the system block |
| Dies on restart | State held in process memory | Durable checkpointing |
| Silent wrong answer | Tool failed and returned an empty result | Explicit failure signalling |
| Unexplainable behaviour | No trace of the assembled prompt or tool calls | Log the rendered request, not the template |
| Cost blowout | History resent every turn with no ceiling | Token and cost budgets |
| Cascading failure | A sub-agent error propagates unchecked | Isolate and bound sub-agent failures |
Not one of these is solved by a better model. This is the central point about why orchestration matters: model capability sets the ceiling on what an agent can do; orchestration determines whether it reliably does it.
Design Principles
- Bound everything. Iterations, tokens, time, cost, depth. The model supplies no limits, so every one must be external.
- Make state durable. Resumability, approval gates and long-running tasks all depend on it.
- Separate decision from execution. The model proposes; orchestration authorises. Never derive permission from model output.
- Fail visibly. A silent failure that produces a plausible answer is worse than a loud one.
- Trace everything. Record the assembled prompt, the response, the tool calls and the outcome.
- Prefer the simplest topology that works. Complexity in coordination costs more than it returns until the task genuinely requires it.
- Design for partial failure. Tools time out and dependencies break; decide in advance what degradation looks like.
Buying Versus Building
Frameworks supply orchestration, with real trade-offs discussed in agent patterns and frameworks. The responsibilities above do not disappear when you adopt one — they are implemented on your behalf, which makes two questions worth asking of any framework.
Which of these does it actually own? Some handle control flow and state well while leaving bounding and observability to you.
Can you see what it does? A framework that assembles prompts and manages state opaquely gives you the responsibilities without the visibility, which is the worst arrangement when something goes wrong.
Key Takeaways
- Orchestration is everything in an agent that is not the model
- Its responsibilities follow directly from the model being stateless, unable to execute and non-deterministic
- It owns control flow, state, context assembly, execution, bounds, recovery and observability
- Bound every dimension — the model supplies no limits of its own
- Durable state is what enables resumability and approval gates
- Orchestration is the only place a record can exist, since the model retains nothing
- Start with a single loop — multi-agent topologies multiply failure modes
- Model capability sets the ceiling; orchestration determines reliability
Frequently Asked Questions (FAQ)
Q: What is agentic orchestration?
It is the layer that coordinates everything around the model — deciding when to call it, managing state between calls, assembling context, executing tools, enforcing limits, handling errors and recording what happened. In short, everything in an agent that is not the model itself.
Q: Why does orchestration matter more than the model?
Because most production failures are orchestration failures — unbounded loops, lost state, silent tool errors, context overflow, missing traces. A better model does not fix any of them. Model capability sets the ceiling on what an agent can do; orchestration determines whether it does it reliably.
Q: What are the main responsibilities of an orchestration layer?
Control flow and termination, state management, context assembly, tool validation and execution, enforcing bounds on iterations and cost, error handling and recovery, observability, and human-in-the-loop approval where actions are consequential.
Q: Why does an agent need iteration limits?
Because the model has no sense of progress and can keep requesting tools indefinitely, particularly when a call fails repeatedly. Without a hard cap the loop does not terminate on its own. The same applies to token, cost, time and recursion-depth limits.
Q: What makes an agent resumable?
Durable state. If conversation history, intermediate results and task progress are checkpointed outside process memory, a run can be paused, survive a restart, or wait for human approval and then continue. State held only in memory is lost on any interruption.
Q: Should I use multiple agents or one?
Start with one. Multi-agent topologies add coordination overhead, duplicate context and allow errors to cascade. The strongest argument for delegation is context isolation — giving a sub-agent its own narrow tool set and history — rather than specialisation for its own sake.
Q: Why is tracing so important for agents?
Because the model retains nothing, so orchestration is the only place a record can exist. Without logs of the assembled prompt, the response and the tool calls, agent behaviour is not debuggable — you cannot inspect a decision that left no trace.
Q: Does using a framework remove these responsibilities?
No, it implements them on your behalf. Worth asking which responsibilities a framework actually owns, since some handle control flow and state well while leaving bounding and observability to you — and whether you can see what it does, because opaque handling gives you the responsibility without the visibility.
Related Reading: