Skip to content

BlogAI

Agent Loop Runaway Costs: Four Bounds That Control Them

Agent Loop Runaway Costs: Four Bounds That Control Them

An agent loop has an unusual shape for a control structure. The body calls a model, the model may call tools, and the loop exits when the model emits something that means done. That puts the exit condition inside the component the loop is supposed to supervise. In an ordinary while loop you write the condition. Here it is authored, on every iteration, by a probabilistic system whose claims of completion you have not verified.

Stories about agent bills, workflows that cost cents in testing and far more at production volume, or loops that ran overnight because nothing stopped them, are anecdotes rather than data. A model-declared done is output, and output gets validated. Termination has to be enforced by controls the loop cannot talk its way past. There are four such bounds, each constraining a different dimension, and they compose rather than compete.

Where the loop is supposed to end

Take one run to trace through the post: an agent asked to rename a module across a repository, update imports, and open a pull request, with tools for editing files, running tests, and calling the code-hosting API. Suppose a well-behaved run takes a dozen model calls.

The model decides it is finished when its own reading of the transcript says so; if a tool call fails with an ambiguous error, it may treat that as success. The orchestrator can verify tool outcomes inside the loop and check completion. The bounds exist to constrain resource use independently of that judgment.

How the run goes wrong

Oscillation and fan-out can contribute to nontermination or an unproductive stop; false success requires separate detection.

False done. The edit tool rejects a write because a file changed underneath it. The model reads the response, concludes the edit landed, and declares completion. The spend is small and the pull request is broken. A cost bound cannot establish correctness; the claimed result needs verification.

Never done. The model keeps discovering one more import to fix, re-reading files, and reasoning about whether it is finished. Each iteration is cheap; there are simply too many.

Two-state oscillation. The model renames the module, tests fail for an unrelated reason, the model reverts, tests still fail, the model renames again. The transcript grows while the repository cycles between two states. Even if the run eventually admits failure, it has spent money without useful work.

Per-step fan-out. A single iteration issues a batch of tool calls, say one file read per module in a large package, and every result lands in the context. The iteration count looks fine; the token count for that step is enormous, and every later model call carries it forward unless pruned.

The four bounds, with their units

Each bound has a unit, and the unit tells you which failure it catches.

  1. Maximum iterations, counted in model calls. Define the counting unit explicitly; framework semantics may differ. For details, see LangChain's max_iterations reference and the OpenAI Agents SDK's run documentation. It caps repeated model calls but does not bound fan-out within one iteration.

  2. Trajectory token budget, counted in cumulative input plus output tokens using the provider's own accounting from each response. An early step with a short context can consume far fewer tokens than a late step carrying forty tool results. This addresses context growth from fan-out and the slow bloat of a long run.

  3. Wall-clock timeout, in seconds, measured from run start rather than per model call. This exists for the person or upstream request waiting on the result. A loop three iterations in but blocked on a tool taking ninety seconds per call can exceed its time budget while staying under the other bounds.

  4. Spend cap, per task and per tenant, in currency, derived from tokens times the provider's current price list plus any metered tool costs. The per-task cap sounds redundant with the token budget until one run mixes models at different prices. A task that costs an acceptable amount, retried by a scheduler or triggered by every row in a tenant's import, can exhaust an aggregate tenant budget.

A usage check after a response detects a threshold crossing; it is not hard admission control. A hard cap requires reserving a defensible upper bound on chargeable work before dispatch, enforcing that allowance, and reconciling actual usage afterward. Tenant accounting must use shared, concurrency-safe reservations across runs over a chosen period, such as a configured billing month. Independent per-run checks can race against the same remaining balance. Without enforceable admission bounds, in-flight charges can overshoot the threshold. That exposure is bounded only if in-flight work has a defensible maximum. Stopping new calls or reaching a deadline does not necessarily cancel already billable work.

None of these substitutes for another. Set each from what it measures: the iteration cap from the task's shape, the token budget from observed trajectories, the timeout from whoever is waiting, the spend cap from the tenant's plan.

What happens at the limit

Before execution, choose an outcome for each bound. There are three defensible ones.

Hard stop with partial state. The run ends and the orchestrator returns whatever the agent produced, labeled incomplete, with the reason the bound fired. In the refactor run, the branch with partial edits is preserved and no new pull-request call is dispatched.

Escalate to a human. The run pauses at a checkpoint, and a person decides whether to raise the bound, redirect, or abandon. Appropriate when the work so far is valuable and what remains is a judgment call.

Graceful failure. The run is discarded, side effects rolled back where possible, and the caller gets a structured error. Appropriate when partial output is worse than none.

Each is a product decision, and the right answer differs by task. Silent truncation, where the loop is cut off and the last model message is returned as if it were the answer, can turn nontermination into false completion.

A configured ceiling is a risk envelope, not a forecast: for example, if your iteration cap is 40 and observed p99 is 11, the gap provides headroom. Monitor usage drift as well as cap hits. Provider context windows and rate limits are not substitutes for trajectory bounds. A context-length error alone does not show that run bounds were absent.

Repetition detection

Oscillation and never-done runs can share a signature: repetition. A loop calling the same tool with near-identical arguments four times may be stuck, but repetition alone does not prove it.

Hash each tool call as tool name plus normalized arguments, and keep recent hashes in run state. Do the same for model output, flagging exactly repeated plans. These are inexpensive exact-repeat signals; they do not generally detect semantic similarity or all near-identical calls. Similarity heuristics are separate and fallible.

A configured repeat threshold can trigger a pause, escalation, or stop. In the refactor run, repeated path and content hashes, combined with repository states and test outcomes showing rename-revert-rename without progress, could identify oscillation. Check outcomes before treating retries or polling as failure. Read-only tools can also loop and still need bounds. Repetition detection can catch some cycles before a resource limit fires and provide a specific reason for stopping.

The failure the loop cannot see

One class of runaway happens to a healthy loop. The code-hosting API degrades, every call returns a 502 after a long delay, and the model retries. Every step spends tokens reasoning about the failure and then waits on a tool that may not recover within the deadline, and concurrent runs do the same.

This is a circuit breaker's job, and it belongs outside the loop. The loop sees one run. A shared breaker sees failures across runs against a tool provider, opens after a threshold, and returns a fast, unambiguous failure to subsequent calls. The orchestrator should treat breaker-open as graceful failure or escalation, with a structured tool-unavailable result, rather than relying on the model to stop retrying.

The bounds also need to survive the orchestrator. Persist the original deadline, iteration count, cumulative tokens, spend, and tool-call hashes so a restarted run does not begin with fresh budgets. Saving after each step still leaves a crash window between billable work and recording its charge. Durable reservations and reconciliation must account for outstanding work before resuming; persistence alone does not prevent duplicate dispatch or double charging.

Verifiers do not replace bounds

The strongest counterargument is that the real fix for false-done is a verifier: an acceptance test, a second model checking the first, or running the test suite before accepting completion. Verification is complementary. Verification can run inside the loop, at checkpoints, or before accepting done. It provides evidence of correctness, subject to the checks' coverage; a second model can also be wrong. A verifier does not itself establish resource bounds.

The caller should be able to distinguish a verified result from incomplete work without having to interpret the model's confidence.

Links to the requested system-design sub-challenge and Aug 13 fan-out post are omitted because their destinations were not supplied or verified.

Share this post