An agent is one loop running over one context window with one set of tools. Splitting it into several agents need not involve a second machine. It gives you separate context windows and a message channel between them, and properties of that channel (serialization, latency, partial failure, ordering) are ones you have already met in distributed systems. So the position this post argues is the same one you would take for any distribution decision in a system-design interview: start with one agent, and split only when tool overload, context exhaustion, or genuinely parallel subtasks produce a measurable benefit that outweighs the coordination, latency, and token cost of the split.

"I'd use a multi-agent architecture" is an architectural choice that needs justification, much like proposing microservices before naming the constraint they address.

The loop and its three budgets

The agent loop is short enough to state in full. The model reads its context, decides whether to call a tool or answer, the tool runs, the result is appended to the context, and the loop repeats until the model stops. Everything that follows comes from the fact that this loop carries state and operates within three budgets.

The first budget is tool-schema tokens. Tool definitions are input on turns that include them, whether or not the model uses the tools. Anthropic's write-up on its tool search feature gives a scale: with a large tool library loaded up front, about 122,800 tokens of context remained for the task, versus about 191,300 when tools were deferred and searched on demand (Anthropic). Those figures describe one vendor's evaluated configuration; schemas occupy context on each turn that includes them.

The second budget is the context window, and it has two limits. The hard limit is the vendor's maximum. The softer limit is the length at which the model still reasons accurately over what it has accumulated. The "Lost in the Middle" study found that retrieval accuracy could degrade when relevant information sat in the middle of a long context in the models and tasks studied (Liu et al., 2023). Treat the advertised window as an admission ceiling and the tested accurate length for your task as the operating limit.

The third budget is wall-clock time, which for a single agent is roughly the time spent on sequential model calls and tool execution. A ten-step task at several seconds per call is slow no matter how good the answer is.

Take a concrete task that will run through the rest of this post: an agent that researches a question across the web and internal documents, then writes a cited report. One agent can do this. Whether it should stay one agent depends on which of the three budgets it exhausts.

Signal one: tool overload

A larger visible toolset can make tool selection less reliable. Anthropic reported internal MCP evaluations where enabling tool search over a large library moved Opus 4 from 49% to 74% accuracy, and Opus 4.5 from 79.5% to 88.1% (Anthropic). These results describe the evaluated models, library, and tasks, rather than a universal tool-count threshold.

Try simpler fixes before splitting agents by tool domain. Group tools into fewer, coarser operations. Retrieve tool definitions on demand rather than loading all of them each turn. Gate tools by phase so the drafting step never sees the search tools. If those measures still leave a step choosing badly, and the misused tools cluster into a domain that would benefit from its own prompt and examples, a specialized agent may be warranted.

For the research agent with one search tool, one document tool, and one write-report tool, there is no tool overload. If it also has twenty connectors to ticketing, CRM, and wiki systems, ask whether tool retrieval fixes selection before you ask whether a "connectors agent" does.

Signal two: context exhaustion

If the research agent reads thirty documents into context, whether they fit depends on document length, tool definitions, task instructions, and accumulated results. Report accuracy may degrade before the window fills.

Still, try the mitigations first. Compaction, meaning periodic summarization of older turns into a shorter note, buys headroom when early details stop mattering. External memory, meaning a scratchpad the agent writes findings to and reads back selectively, converts context from an accumulation problem into a retrieval problem. Both keep a single loop and a single decision-maker.

A subagent can help when a subtask's context should be isolated rather than merely shortened. Reading one dense document and extracting the five relevant facts is such a subtask. The reading agent consumes the full document in its own window, returns a compact result, and its window is discarded. The parent retains the extraction; the worker's document processing remains part of total system cost. This is the case Anthropic describes in its multi-agent research system, where subagents each explore a slice of a research question and return condensed findings to a lead agent (Anthropic). The context benefit is measurable: parent context per document can drop from the document's length to the summary's length.

Signal three: genuinely parallel subtasks

A single agent can issue several tool calls in one turn and receive the results together, so if the parallel work is "run these five searches," parallel tool calls handle it inside one loop without separate agents.

Independent multi-step subtasks are stronger candidates for separate agents. Favor work without conflicting writes; partial dependencies can still permit overlap, and parallel reasoning can feed coordinated or serialized writes. Dependencies and shared writes constrain overlap and require coordination; uncoordinated execution can produce stalls or conflicts. Measure whether critical-path savings exceed delegation, synchronization, and aggregation overhead. Compare end-to-end latency, total tokens, and actual cost against the single-agent baseline at comparable output quality.

Cognition's "Don't Build Multi-Agents" essay offers a counterposition here (Cognition). Its argument is that every action an agent takes encodes implicit decisions, and parallel subagents that cannot see each other's decisions can make conflicting ones the parent then has to reconcile; their example is two subagents building parts of a game with incompatible visual styles. Independent research reads can be easier to reconcile than coupled design decisions. Parallel code work can still be appropriate with clear interfaces and coordinated integration.

What the coordinator costs

A supervisor that decomposes a task and aggregates the results is itself an agent making model calls. Delegation briefs consume supervisor tokens, though one model call can prepare several delegations; returned results consume context, and final synthesis adds tokens and calls. Total tokens include worker and supervisor usage. Context isolation can reduce repeated input, while batching and model pricing affect the actual cost relative to a single-agent baseline. Account for communication latency separately.

Anthropic's reported measurements give a sense of scale: single agents used roughly 4× the tokens of a chat interaction and the multi-agent research system roughly 15× (Anthropic). Those ratios describe its system and task mix; the task's value has to justify the measured cost.

Latency depends on topology. Fan-out reduces elapsed time only if it shortens the critical path enough to cover coordination overhead. The supervisor's decomposition and aggregation add serial work. A sequential pipeline adds stage transitions, though smaller contexts or different models may offset that latency. The supervisor can accumulate every worker's output, so context exhaustion can reappear there. Workers should return bounded findings rather than transcripts. Designing the return contract, meaning what a worker may send back and how long it may be, is a central part of keeping the coordinator healthy.

Pick topology by task shape. Fixed stages with real data dependencies, such as search, then extract, then draft, then review, are a sequential pipeline, and a single agent with phase-gated tools often handles them without any split. Independent parallel reads with a merge step are fan-out. Dynamic decomposition, where the number and nature of subtasks are unknown until the task is under way, is supervisor-and-worker. These patterns can combine: a supervisor may fan out independent reads within a larger pipeline.

Where agents collide over shared state

Once two agents can write to the same store, the failures are the ones you know from concurrent systems, wearing new names. Two agents fetch the same record, both modify it, and the second write erases the first: a lost update. In a store permitting uncommitted reads, one agent reads another's draft edits before the writer's transaction rolls back: a dirty read. Reading an older committed version is a stale read. One agent waits for a signal from another that already errored and exited: an orphaned waiter, which without a timeout waits forever. Two agents each claim the same source document: duplicated work at best, contradictory summaries at worst.

The mitigations are also the ones you know. Give each shared artifact a single writer; in the research example, only the lead agent edits the report, and workers return findings rather than edits. Make worker operations idempotent so a retried subtask does not double-append. Put timeouts on every delegation so the parent never blocks indefinitely on a dead child. Where two agents must touch the same resource, use a claim or lease at the store boundary rather than trusting prompts to keep them apart; a prompt is not a mutex.

Naming these as concurrency failures in the interview makes the state and coordination requirements explicit.

The interview answer

When asked to design an agentic system, the answer that holds up begins with one agent and earns each split. State the loop and its budgets. Describe the single-agent version, using compaction, external memory, tool retrieval, or parallel tool calls where warranted. Then name the signal that would make you split, the simpler mitigation you would try first, and the measurement that would show the split paid off: tool-selection accuracy on an evaluation set, parent context consumed per subtask, or elapsed time against a latency target with tokens and actual cost beside it.

For the research agent, the honest answer is probably one agent with phase-gated tools and a scratchpad. Introduce document-reading subagents only when accumulated reading makes the single-window baseline inaccurate despite appropriate context mitigations. Compare parent-context tokens per document and report accuracy with the single-agent baseline before keeping the split. Only the lead agent writes the report; workers return findings.

Three-signal split checklist

1. Tool overload. Is a step choosing tools badly with the full set visible? Try grouping, on-demand tool retrieval, and phase gating first. Consider a split when a domain-specific agent improves evaluated tool-selection accuracy.
2. Context exhaustion. Does one task exceed the tested accurate length, not just the vendor limit? Try compaction and external memory first. Split when isolating a subtask and returning a summary reduces parent context without reducing answer accuracy.
3. Genuinely parallel subtasks. Favor independent multi-step work without conflicting writes; coordinate partial dependencies and shared writes. Try parallel tool calls first. Keep the split only when measured latency gains justify coordination overhead, checking total tokens and actual cost at comparable quality.

If none of the three applies, keep one agent and explain how it meets the requirements.