multi-agent-orchestration
verified3f10425c-5131-435a-add8-9b86a3fa05e9
Design and run multi-agent systems that actually scale — supervisor vs swarm vs peer patterns, shared state, timeouts/spend caps, and error handling.
Metadata
Skill file
# Multi-Agent Orchestration
Use when one agent can't do the job — the task needs **specialists** (each with its
own tools/knowledge/roles), parallelizable subtasks, or a reasoning/plan/review
loop. This is where most single-agent apps plateau (around ~15 tools) and where
architecture choice determines whether coordination helps or tanks performance.
## Pick the coordination pattern for the *task*, not the hype
Five dominant patterns; the mistake is choosing one and forcing every workload
through it:
1. **Supervisor / orchestrator-worker.** A central coordinator (supervisor) plans,
routes work to specialist workers, monitors them, and merges results. Best for
tasks needing **dynamic routing**, ordered execution, and conflict resolution.
*Caution:* the supervisor becomes a bottleneck, and the pattern can degrade fast
parallel subtasks (reported: helped parallel tasks ~+80%, harmed sequential
reasoning ~−70%).
2. **Hierarchical.** Nested supervisors (manager → sub-managers → workers). Good for
large, decomposable organizations of agents, but adds depth/latency.
3. **Peer-to-peer / swarm.** Peers exchange work and hand off sequentially through
distributed control, no central controller. Fits **independent, strictly-ordered**
subtasks where routing is embedded in the task (e.g. handoff chains). OpenAI's
Swarm popularized this (`Agent` + handoff primitives); its successor is the
OpenAI Agents SDK. *Caution:* using a swarm for work needing coordination/ordering
makes agents drift into contradictory outputs.
4. **Blackboard.** Agents share a common workspace/memory and react to what's written;
good when no fixed orchestration is known up front. Complex to reason about.
5. **Agent-as-tool / graph.** One agent calls another agent *as if it were a tool*, or
agents are wired into an explicit execution graph/workflow (also where A2A — an
open cross-org agent communication protocol — comes in for interoperability).
**Rule of thumb:** embarrassingly-parallel work → fan-out workers under a supervisor;
strict sequential handoffs → swarm/chain; dynamic multi-step planning → supervisor;
explicit, reproducible pipelines → graph/workflow engine.
## Best practices that separate production from demos
- **Keep roles minimal and sharply defined.** plan / worker / reviewer is usually
enough; every extra role adds coordination overhead. Tight, domain-focused system
prompts per agent; broad "do anything" agents collide.
- **Drive control with a workflow engine, not the LLM.** Use explicit state machines
(steps, transitions) rather than free-form chat for retries, timeouts, and
idempotency. Don't let agents improvise the control flow.
- **Shared memory, not peer pinging.** Have agents read/write the same memory/state
store instead of messaging each other through the LLM — cheaper and more reliable.
- **Add an evaluator gate.** A review/validation step that kills bad plans and enforces
stop conditions before wasting tool calls.
- **Specialists via tools, not raw agents.** An orchestrator can expose each worker as
a *tool* with a clear docstring/description so the supervisor knows when to call it.
- **Cost control is non-negotiable.** Set hard caps on **tokens, steps, and spend** per
run; multi-step chains blow budgets fast with no guardrails.
## Hard requirements (state, guards, observability)
- **Timeouts and step limits** on every agent and the whole run — a runaway loop is
the #1 production failure in multi-agent systems.
- **Idempotent tool calls**, especially for writes — retries must not double-charge
or double-create.
- **Log every message with a trace/replay id** and a consistent schema. Multi-agent
debugging is impossible without the ability to replay the exact conversation and
tool calls (see the LLM-observability skill).
- **A stop condition / success criteria** per agent and overall, plus a global escape
hatch.
## Pitfalls
- **Orchestrator as throughput bottleneck** for parallel work — fan out, don't funnel.
- **Undisciplined LLM-driven control flow** — nondeterministic, unretryable, hard to
debug. Prefer explicit workflows/state machines.
- **Agents duplicating each other** because roles/contexts overlap — assert ownership
of specific tools/data per agent.
- **No budget caps** → a small task turns into hundreds of calls.
- **Coordinator single-point-of-failure** / sequential reasoning slowness in
supervisor patterns.
- **Over-engineering** — most tasks don't need many agents; a single well-designed
agent + tools wins for simple workflows. Multi-agent is justified by real role
separation or parallelism, not as a default.
## Verify
- Each workload's chosen pattern actually improves correctness/latency vs a single
agent on an eval set (don't assume).
- Runs respect caps: max steps, max tokens, max spend — enforced, logged, alertable.
- A parallelizable task completed in near wall-clock of its slowest worker (not
serialized through a bottleneck).
- A failing step triggers a clean, idempotent retry and a replayable trace.