Multi-Agent Systems

A single LLM agent (a model in a loop, calling tools until a task is done) handles a surprising range of work. But some tasks are too broad for one context window, too slow done one step at a time, or benefit from separate perspectives. Multi-agent systems split the work across several model instances that coordinate. They power deep-research products, large code migrations and some customer-service architectures. They also multiply cost, add new failure modes, and are often adopted when a single well-built agent would do better. This lesson teaches you to tell the two situations apart and to build the multi-agent version properly when it is warranted.

The intuition: a newsroom, not a crowd

A newspaper does not put forty reporters in one room and let them all edit one document. It has an editor who decides the stories, assigns reporters, each reporter researches independently with their own notes, and copy editors check the drafts. Information flows through defined channels: the assignment, the filed story, the edit notes.

The things that make a newsroom work are the things that make a multi-agent system work:

• Specialization: each role has its own instructions and tools.
• Parallelism: reporters work at the same time.
• Context isolation: a reporter does not need every other reporter's notes, only the assignment. This keeps each person's working memory focused.
• A clear owner of the whole: the editor integrates and is accountable.

What breaks newsrooms also breaks agent systems: vague assignments, two people unknowingly writing the same story, contradictory facts that nobody reconciles, and meetings that never end.

Why use more than one agent

There are three genuine reasons, and it pays to name which one you are relying on.

1. Context isolation. An agent's context window is its working memory. A research task that reads 200 web pages cannot fit them all in one context, and even when it can, quality degrades as irrelevant material accumulates. A subagent can read 20 pages, then return a 500-token summary. The lead agent sees only the summaries. In effect, subagents act as context compressors.

2. Parallelism. Independent subtasks (research five competitors, test five modules) can run concurrently. Wall-clock time drops from the sum of subtask times toward the maximum.

3. Specialization. Different roles can have different system prompts, tools, permissions and even models: a cheap fast model for classification, a strong model for planning, a sandboxed agent with code execution, a read-only agent for sensitive data. Narrow tool sets also make each agent's choices easier and safer.

A fourth reason is often claimed, "diverse perspectives", and it is real but weaker than it sounds, because instances of the same model share the same blind spots. More on that under debate.

Core patterns

The vocabulary below follows common usage, including Anthropic's December 2024 engineering essay "Building effective agents", which distinguishes fixed workflows (code decides the control flow) from agents (the model decides).

Pipeline. Each stage is an LLM call with its own prompt; code passes the output along, optionally with a validation gate between stages. This is the most predictable pattern and often all you need. It is a workflow, not really a team of agents.

Orchestrator–workers. A lead agent reads the task, writes subtask specifications, spawns workers (often in parallel), receives their results and decides whether to spawn more or to synthesize. The quality of the system is dominated by the quality of the subtask specifications: each must state the objective, the expected output format, the tools to use, and the boundaries ("cover only 2024 onward; do not research pricing, another worker is doing that").

Debate and critique. A generator produces an answer, a critic finds problems, the generator revises. Du et al. (2023) reported that having several model instances propose answers and debate over rounds improved reasoning and factual accuracy on their benchmarks. The pattern works best when critique can be grounded in something external, such as running tests, checking a citation or executing the code, rather than one model's opinion of another's prose.

Hierarchical. Orchestrators that manage sub-orchestrators. Each level summarizes, so information is lost at each hop. Use only when the task truly has that depth.

Swarm / handoffs. No central controller: a triage agent hands the conversation to a billing agent, which may hand it to a refunds agent. Each handoff is implemented as a tool call that returns the next agent. Simple and natural for customer-service routing; prone to loops without a hop limit.

Communication and shared state

Agents communicate in one of two ways:

• Message passing. Agent A's output becomes part of agent B's input. Simple and auditable, but every hop re-serializes knowledge into text, and nuance is lost.
• Shared state (blackboard). Agents read and write a common store: a task board, a file system, a database, a shared document. Workers can write full results to storage and pass back only a reference, avoiding the "game of telephone" through the orchestrator.

Design rules that hold across frameworks:

1. Typed messages. Subtask specs and results should be structured (see the structured outputs lesson): objective, constraints, status, findings, sources, open questions.
2. Single writer per item. Two agents editing the same file or record concurrently produces conflicts that LLMs reconcile badly. Partition work so each artifact has one owner, or serialize writes.
3. Shared decisions, not just shared data. Cognition's 2025 essay "Don't Build Multi-Agents" argues that actions carry implicit decisions, and parallel workers that cannot see each other's decisions produce incompatible pieces (their example: one subagent builds a background in one visual style while another builds a character in another). Either share the full trace with each worker or restrict parallel work to tasks that are genuinely independent.
4. Explicit termination. Every loop needs a stopping rule in code: maximum turns, maximum spend, maximum wall time.

Agent-to-agent protocols

Within one application, agents are just function calls. Across organizations or vendors you need a protocol. Two are worth distinguishing:

• Model Context Protocol (MCP) connects an agent to tools and data (covered in its own lesson). The server exposes capabilities; the agent is the client.
• Agent2Agent (A2A) connects an agent to another agent treated as an opaque peer. Google announced A2A in April 2025 and contributed it to the Linux Foundation in June 2025 as an open-source project with multi-vendor governance. Its core ideas: each agent publishes an Agent Card (a JSON document describing its identity, skills, endpoint and authentication requirements); a client agent sends a task to a remote agent over HTTP using JSON-RPC; tasks have a lifecycle (submitted, working, input-required, completed, failed and so on) and can stream updates or deliver them by push notification; results come back as artifacts composed of typed parts (text, files, structured data). The remote agent does not expose its internal tools or memory, which is the point: you can delegate to an agent you do not control.

The protocol has moved quickly through versions since launch. Before building on it, read the current specification and check which version your counterparties implement. Concepts such as agent cards, task lifecycles and opaque delegation are stable; field names and transport options are less so.

Security note: a remote agent is an untrusted party. Its outputs are as dangerous as any web page. Authenticate it, scope what you send it, and treat what it returns as data, not instructions.

Failure modes, with numbers

Error cascades

In a pipeline of $n$ stages where stage $i$ succeeds with probability $p_i$ and errors are not caught, end-to-end success is

Five stages at $p = 0.95$ give $0.95^5 = 0.774$. Almost one run in four fails, though every stage looks fine in isolation. Worse, downstream agents tend to accept upstream errors because they arrive formatted as facts. Mitigations: validation gates between stages, workers that report confidence and sources, and a final verifier that checks claims against evidence.

The limits of voting and debate

Suppose three agents answer independently, each correct with probability $p = 0.7$, and you take the majority. The majority is correct if all three are right or exactly two are:

Improvement from 0.70 to 0.78 looks good, but it assumes independent errors. Three copies of the same model with the same prompt make correlated mistakes; in the limit of perfect correlation, the vote is just one agent at triple the price. Diversity of model, prompt, tools or evidence is what buys real gains.

Cost explosion

Each agent re-reads its own context on every turn. If a lead agent spawns $k$ workers that each run $T$ turns over an average context of $C$ tokens, worker input tokens alone are roughly $k \cdot T \cdot C$, before the lead's own turns. Example: 6 workers × 12 turns × 20,000 tokens = 1.44 million input tokens for one user query. Anthropic's June 2025 write-up of its multi-agent research system reported that its agents used about 4 times as many tokens as chat interactions, and multi-agent runs about 15 times as many; the same post reported that, on its internal research evaluation, a multi-agent configuration (a Claude Opus 4 lead with Claude Sonnet 4 subagents) outperformed a single Claude Opus 4 agent by 90.2%. Their conclusion is the right one: multi-agent architectures pay off on high-value, breadth-heavy tasks, and are uneconomical for most others.

Coordination loops and other pathologies

• Ping-pong handoffs: agent A hands to B, which hands back to A.
• Endless critique: the critic always finds something; the generator always revises.
• Duplicate work: two workers given overlapping specs research the same thing.
• Over-spawning: the orchestrator spawns 50 workers for a question needing 2.
• Lost context on handoff: the receiving agent lacks a detail the user gave earlier.
• Premature termination: the orchestrator declares success with gaps.

A 2025 study by Cemri et al., "Why Do Multi-Agent LLM Systems Fail?", analysed traces from several open-source frameworks and grouped failures into specification problems, inter-agent misalignment, and weak task verification. That is a useful checklist for your own post-mortems.

Code: an orchestrator with hard limits

import json
from concurrent.futures import ThreadPoolExecutor

def call_llm(system: str, user: str) -> str:
    raise NotImplementedError("wire to your provider")

class Budget:
    def __init__(self, max_calls: int):
        self.left = max_calls
    def spend(self):
        if self.left <= 0:
            raise RuntimeError("budget exhausted")
        self.left -= 1

def plan(task: str, budget: Budget) -> list[dict]:
    budget.spend()
    raw = call_llm("Split the task into at most 4 independent subtasks. Return a JSON list "
                   'of {"objective", "scope", "output_format"}.', task)
    return json.loads(raw)[:4]                      # enforce the cap in code too

def work(spec: dict, budget: Budget) -> dict:
    budget.spend()
    out = call_llm("You are a research worker. Stay within scope. Return JSON "
                   '{"findings": [...], "sources": [...], "gaps": [...]}.', json.dumps(spec))
    return json.loads(out)

def orchestrate(task: str, max_calls: int = 10) -> str:
    budget = Budget(max_calls)
    specs = plan(task, budget)
    with ThreadPoolExecutor(max_workers=4) as pool:
        results = list(pool.map(lambda s: work(s, budget), specs))
    budget.spend()
    return call_llm("Synthesize the findings. Flag contradictions and gaps explicitly.",
                    json.dumps({"task": task, "results": results}))

What matters here is not the framework but the controls: the number of subtasks is capped in code (not only in the prompt), every model call draws from a shared budget, workers return structured results with sources and gaps, and the synthesizer is told to surface contradictions rather than smooth them over. In production, add validation of each worker's JSON, timeouts, and tracing of every call with a shared run ID. (The simple counter is not thread-safe; use a lock in real code.)

A handoff loop needs its own guard:

def run_with_handoffs(agents: dict, start: str, message: str, max_hops: int = 5) -> str:
    current, seen = start, []
    for _ in range(max_hops):
        reply, next_agent = agents[current](message)   # each agent returns (text, handoff or None)
        seen.append(current)
        if next_agent is None:
            return reply
        if seen.count(next_agent) >= 2:                 # crude ping-pong detection
            return "Escalating to a human: routing loop detected."
        current = next_agent
    return "Escalating to a human: too many handoffs."

When a single agent is better

Both of the engineering write-ups cited above, which reach opposite-sounding headlines, agree on the core point: multi-agent systems help when subtasks are independent and context-heavy, and hurt when subtasks are tightly coupled. Anthropic noted that domains needing all agents to share the same context, or with many dependencies between agents, were a poor fit for their parallel design at the time, and that most coding tasks involve fewer truly parallelizable subtasks than research. Cognition argued that for tightly coupled work like writing software, a single-threaded agent with good context management (including compressing older history) is more reliable.

A practical decision rule:

1. Start with one agent and good tools. Measure.
2. If it fails because the context overflows with material that could be summarized, try subagents as context compressors.
3. If it is too slow on independent subtasks, parallelize those subtasks only.
4. If it fails because roles need different permissions or tools, split along that boundary.
5. If the subtasks need to agree on many small decisions, keep them in one agent.

Evaluating multi-agent systems

Evaluate at two levels.

End-to-end: task success rate, quality (rubric-based LLM judge plus human spot checks), cost per task, latency (median and tail), and the comparison that justifies the architecture: the same tasks run by your best single agent at the same budget. If the multi-agent system does not beat it on quality per dollar, you have built complexity for nothing.

Component and trace level: decomposition quality (are subtasks complete and non-overlapping?), worker success rates, handoff accuracy (did triage route correctly?), loop and timeout frequency, and how often the synthesizer drops or distorts worker findings. Multi-agent runs are non-deterministic, so evaluate over many runs and report variance. Tracing every call with parent–child links is not optional; without it you cannot debug a failure that started three agents upstream.

Common pitfalls

• Starting multi-agent. Building the team before proving a single agent cannot do the job.
• Vague subtask specs. "Research the market" produces overlapping, unfocused work. Specify objective, scope, exclusions and output format.
• Limits only in the prompt. "Spawn at most 3 workers" must also be enforced by code.
• Parallelizing coupled work. Workers making independent decisions on a shared artifact produce incoherent results.
• Trusting consensus. Agreement among copies of one model is weak evidence; ground critique in tests and sources.
• No shared run ID. Without trace correlation, debugging is guesswork.
• Treating remote agents as trusted. A2A peers and other agents' outputs are untrusted input.

In practice

• Prefer workflows (fixed code-driven pipelines) when the steps are known; use agentic orchestration only for open-ended decomposition.
• Use orchestrator–workers for breadth-first, read-heavy tasks: research, auditing many documents, surveying a codebase. Give workers narrow tools and require structured, sourced outputs.
• Use generator–verifier loops where verification is objective (tests, schema checks, citation lookups), with a capped number of rounds.
• Use handoffs for conversational routing, with a hop limit and a human escalation path.
• Budget everything: calls, tokens, wall time, and depth. Alert on runs that hit limits.
• Keep a single-agent baseline in your evaluation suite and re-run it whenever models improve; stronger models frequently make yesterday's multi-agent design unnecessary.