The Honest Decision Framework
35 min readreadingWhen Multi-Agent Wins
2 of 13Multi-Agent Systems Engineering
Up next · Supervisor-Workers and Hierarchies
The postmortem is uncomfortable reading. A team spent six weeks building an eight-agent "research swarm" — planner, four searchers, a synthesizer, a fact-checker, a formatter. Demo day went well. Then the invoice arrived: token spend up 23x over the single-agent baseline, median latency up from 40 seconds to 4 minutes, and when they finally ran a blind quality eval, the swarm tied the baseline on 70% of queries and beat it on 12%. Nobody had run the eval before building, because the architecture felt obviously better. It felt like a team.
This lesson is the framework that postmortem was missing. It has three parts: the four situations where multi-agent genuinely wins, the three bills it always presents, and the discipline — the null hypothesis — that forces the decision to be made with data instead of aesthetics. It closes by working the decision on three concrete workloads, because a framework you cannot apply in ten minutes is a poster, not a tool.
Context-window pressure. The task's working set does not fit one window — or fits so tightly that quality degrades as the window fills. A deep research task that must read 60 sources cannot hold them all in one context; ten subagents reading six sources each hold them in parallel windows and return compressed findings. This is the clearest win because it is physical: no amount of prompt engineering gives a single agent eleven context windows. Anthropic's research system is built on exactly this — the lead agent's plan fans out into subagents precisely so the reading happens in windows the parent never pays for.
Genuinely parallelizable subtasks. The work decomposes into pieces that do not depend on each other's outputs: analyze eight competitors, review 40 files for a deprecated API, evaluate one candidate answer per rubric criterion. Wall-clock time drops toward the slowest child instead of the sum. The test is strict: if subtask B needs subtask A's conclusions, you have a pipeline, not a fan-out, and the parallelism is imaginary.
Capability specialization. Different subtasks genuinely want different models, prompts, or tool sets — a cheap fast model for extraction, an expensive one for synthesis; a code-focused prompt for the reviewer, a prose-focused one for the writer. Specialization also shows up as judgment independence: a critic that did not produce the draft. Note the honest version of this win is often cost shaped: the fleet of cheap extractors under an expensive synthesizer can beat one big model on quality per dollar.
Permission separation. Security boundaries you can only draw between agents, not within one. The agent that reads the public internet must not be the agent holding deploy credentials — if one prompt holds both, a prompt injection in a fetched page is one hop from production. Splitting the roles puts a reviewable, permission-checked handoff between untrusted input and privileged action. This win is unique on the list: it can justify a second agent even when quality and cost say no, because the alternative is a security posture, not a performance number.
Now the costs, with arithmetic instead of adjectives. Anthropic reports that agents use about 4x the tokens of chat interactions, and multi-agent systems about 15x. It is worth seeing where the 15x physically comes from, because each term is a design decision you can push on:
Scale it: at 10,000 queries a day, the chat workload is 20M tokens; the multi-agent version is 300M. At Claude Opus 5 input pricing ($5 per million tokens), the difference is roughly $1,400 a day on input alone — before output tokens, which are 5x the price. The multiplier is not a reason never to build multi-agent; Anthropic's own conclusion was that the spend is precisely why the architecture wins on hard research tasks, since it buys parallel context windows. The multiplier is a reason to reserve the architecture for queries whose value supports it — which is what effort scaling in Lesson 4 is for.
Latency first, because it cuts both ways. Fan-out genuinely reduces wall-clock time on parallel work — that is its point. But every sequential handoff adds a full model round-trip, and a fan-in barrier waits for the slowest child. A pipeline of four agents at 30 seconds each is two minutes before any retry. Multi-agent systems are usually the right choice for background, minutes-scale work and the wrong choice inside an interactive request-response loop.
Error compounding is the quieter bill. Model calls are not reliable steps; suppose each stage of a pipeline does the right thing 90% of the time. Chain five stages and end-to-end reliability is:
Fifty-nine percent. A coin flip and change, from stages that each look fine in isolation. And unlike traditional distributed systems, an early error does not usually crash the pipeline — it propagates, laundered into fluent text that downstream agents treat as ground truth. The table makes the shape of the problem clear:
| Per-step reliability | 5 steps | 10 steps | 20 steps |
|---|---|---|---|
| 0.99 | 0.95 | 0.90 | 0.82 |
| 0.95 | 0.77 | 0.60 | 0.36 |
| 0.90 | 0.59 | 0.35 | 0.12 |
Two design consequences. First, keep chains short: every stage you remove is a multiplication you skip. Second, if you must be long, insert verification — a checker agent, a schema validation, a test suite — because a 0.90 stage followed by a checker that catches 80% of its errors behaves like a 0.98 stage. This is why critic and verifier agents keep appearing in successful systems: they are reliability engineering, not ceremony.
A single agent has one prompt, one loop, one trace to read. N agents have N prompts, N loops, and up to N(N-1)/2 interaction edges, any of which can host a failure that exists in no agent individually — the MAST taxonomy's inter-agent misalignment category, roughly a third of observed failures. When a multi-agent run goes wrong, you are debugging a distributed system whose messages are natural language: nondeterministic components, lossy channels, and no stack traces across the boundary. Budget for observability from day one — traces that record every handoff payload — or accept that failures will be unexplainable.
This is also the honest place to register the strongest published counterargument. Cognition's 2025 essay Don't Build Multi-Agents argues that fragmenting context is the original sin: every agent boundary loses implicit context and unspoken decisions, and a single agent with a carefully managed, continuous context avoids the entire class of coordination failures. Their conclusion — context engineering beats agent multiplication for most tasks, especially coding, where actions carry implicit decisions that conflict when made in parallel — is the null hypothesis of this course stated by someone who ships on it. The two camps agree more than the headlines suggest: Anthropic's system succeeds on read-heavy, parallelizable research; Cognition's warning is sharpest for write-heavy, tightly coupled work. Knowing which kind of workload you have is most of the decision.
The framework compresses to one sentence: a single agent with good tools is the null hypothesis, and the second agent must reject it with data. In practice:
Check: your single-agent baseline scores 71% on your eval. A proposed three-agent pipeline scores 74% at 6x the tokens and 3x the latency. The framework's verdict is:
Workload A — competitive landscape research. "Profile our 8 main competitors: pricing, recent launches, hiring signals; produce a comparison brief." The working set is dozens of web sources — past one window. The eight profiles are independent — genuinely parallel. Extraction is cheap-model work; synthesis wants a strong model — specialization. Verdict: multi-agent fan-out, 8 workers, one synthesizer. Three of four wins apply; this is the Anthropic-research-system shape, and the 15x is buying real parallelism.
Workload B — migrate a codebase off a deprecated ORM. Forty files, but edits interlock: shared models, call sites that must change together, a test suite that gates every step. Parallel editors would each make locally reasonable, globally conflicting decisions — Cognition's exact warning. The working set (the relevant slice of the repo) fits a modern window with good retrieval. Verdict: single agent, strong tools, continuous context. The null hypothesis survives. If anything fans out, it is read-only analysis of which files need changes — not the edits.
Workload C — support assistant that can issue refunds. Quality-wise a single agent handles triage fine; the working set is small. But the agent reads arbitrary customer text — untrusted input — and one of its tools moves money. Verdict: two agents, split at the permission boundary. A conversational agent with read-only tools drafts a structured refund request; a separate executor agent with the privileged tool validates it against policy and executes. The second agent is not there for quality; it is there so that no prompt containing customer text also contains the refund capability. Security wins can overrule the null hypothesis; they are the only wins that can do it without an eval.
| Workload | Window pressure | Parallel | Specialization | Permission | Verdict |
|---|---|---|---|---|---|
| Competitor research | Yes | Yes | Yes | No | Fan-out + synthesizer |
| ORM migration | No | No (coupled) | No | No | Single agent |
| Refund assistant | No | No | No | Yes | Two agents at the boundary |
What you can now do: screen any proposed multi-agent design in ten minutes, price its three bills with real numbers, and demand the A/B that makes the decision for you. The rest of the course assumes this discipline — every topology and mechanism that follows is for workloads that passed this screen.