Loop engineering — designing the agent's iteration, not its prompt.

"Loop engineering" is the discipline that comes after prompt engineering and context engineering. A modern AI agent isn't a single prompt — it's a loop: the model thinks, calls a tool, sees the result, and repeats until done. Loop engineering is the craft of designing that loop well — how context flows in and out, how the loop decides to stop, how it recovers from errors, and how it avoids running forever without making progress. Andrew Ng frames the bigger picture as nested loops (fast coding loops inside slow feedback loops); this article works from the innermost loop outward, with the concrete engineering — message-history mechanics, the multiple-exits termination doctrine, error-as-observation, and the loopmaxxing trap — that separates a reliable agent from one that burns tokens in circles.

In mid-2026 the phrase caught fire. Andrew Ng noted that "loop engineering" had become a hot buzzphrase; Jensen Huang was quoted saying prompt engineering is becoming obsolete and loops are the new paradigm. Behind the hype is a real shift. For two years the skill was writing a good prompt — a single, careful instruction. Then it was context engineering — assembling the right documents, tools, and memory into the model's window. Now the frontier is loop engineering: you no longer type each next instruction by hand; you build a program that prompts the agent, checks its output, updates state, and re-runs it — over and over — until a goal is met. The unit of work moved from the message to the iteration.

This is a technical article, not a think-piece. We'll build the concept from the message level up: what one turn of an agent loop actually contains, how the conversation grows and must be managed, exactly how the loop should decide to stop, how errors become part of the loop rather than crash it, and the failure modes — chief among them "loopmaxxing" — that show up in production. Then we'll zoom out to Ng's nested-loop model and a rollout playbook.

What loop engineering is (and what it replaced)

Three eras of "the skill," each a strict superset of the last:

Era	Unit of work	You control…	Failure looks like…
Prompt engineering	one message	the wording of a single instruction	a bad single answer
Context engineering	one window	what documents, tools, memory the model sees	missing/again-irrelevant context
Loop engineering	one iteration cycle	how the model is invoked repeatedly, and when it stops	infinite loops, runaway cost, silent non-progress

The clean definition: loop engineering is designing the program that prompts, checks, remembers, and re-runs an agent — instead of you typing every next instruction by hand. The agent's behavior depends far less on any single clever prompt than on the architecture of the loop: exactly how the LLM is called in a cycle to reason and act, and — the part beginners miss — exactly how and when that cycle ends. Prompt quality still matters, but it's now one input to a system, not the system.

The anatomy of a single agent loop

Start at the smallest scale: one agent, one loop. Nearly every framework — LangChain, the OpenAI Agents SDK, Claude's agent harness, Letta, Google's ADK — implements the same six-line skeleton. It is worth memorizing because everything else is a refinement of it:

messages = [system_prompt, user_goal]
while not done:
    response = call_llm(messages, tools)      # 1. invoke the model
    messages.append(response)                 # 2. record what it said/decided
    if response.tool_calls:                   # 3. did it want to act?
        results = execute(response.tool_calls)# 4. run the tools
        messages.append(results)              # 5. feed results back
    else:
        done = True                           # 6. no tool call = it's finished
return response

That loop is the entire mechanism of an "AI agent." The model is called with the running conversation plus a list of tools. It either emits a final answer (loop ends) or asks to call one or more tools. Your code runs those tools, appends the results to the conversation, and calls the model again — now one observation richer. The model reasons over the accumulated history each turn and decides the next move.

The ReAct pattern underneath

The intellectual root is ReAct (Reason + Act), from Yao et al., 2022. Its insight was to interleave two things the model produces at each step: a reasoning trace ("I should check the file exists before editing it") and a concrete action (the tool call). Crucially, "the observation from each action feeds into the next reasoning step" — so the agent adapts to what actually happened instead of blindly executing a pre-made plan. Modern agent loops are ReAct with better tooling: the reasoning is the model's chain-of-thought, the action is a structured tool call, and the observation is the tool result appended to history.

That six-line skeleton is deceptively simple. The gap between the toy version and a production agent is entirely in the details we cover next: how the message history is managed so it doesn't explode, how the loop is guaranteed to terminate, and how errors are absorbed rather than fatal.

The message history: where the tokens (and bugs) live

Each iteration appends to the conversation, so the loop's context grows monotonically. Understanding that growth is half of loop engineering, because it drives both cost and correctness.

The exact append order matters

A subtle, extremely common bug: the assistant message that contains the tool calls must be appended to history before the tool-result messages, and each tool result must reference the matching tool_call_id. The API rejects a tool-result message that references a tool_call_id it hasn't seen. Get the ordering wrong and you get a validation error that doesn't obviously point at the cause.

# CORRECT ordering
messages.append(assistant_msg)              # has tool_calls: [{id: "call_7", ...}]
for call in assistant_msg.tool_calls:
    result = execute(call)
    messages.append({                        # tool result MUST come after,
        "role": "tool",                      # and reference the same id
        "tool_call_id": call.id,             # "call_7"
        "content": str(result),
    })

Tool results dominate the token budget

Here's a number that reshapes intuition: in real agent traces, tool responses account for roughly two-thirds of total tokens (one analysis put it at 67.6%). The model's own text is a minority; the bulk is what tools hand back — file contents, API JSON, search results, error logs. This has a direct engineering consequence: the highest-leverage way to control an agent's cost and keep its context coherent is to control what tools return, not to trim the prompt.

Three context-management techniques inside the loop

• Paginated / windowed reads. Don't let a tool dump a 5,000-line file into context. Return a window (e.g. 200 numbered lines) and let the agent request more. The line numbers double as addresses for later edits.
• Surgical edits, not rewrites. When the agent modifies a file, have it emit a targeted diff (change lines 40–48) rather than re-emitting the whole file. Less output, less context, fewer chances to corrupt untouched code.
• Compaction and isolation. At roughly 80% context utilization, trigger compaction: summarize the older turns into a condensed state and continue from the summary. For big sub-tasks, spawn a sub-agent with its own fresh context that returns only a short result to the parent — isolating the sub-task's token churn from the main loop. (This is the "orchestrator + workers" pattern many multi-agent systems use.)

These map directly onto the KV-cache and memory realities of inference: a longer context is more expensive to serve and slower to attend over, which is exactly why compaction pays off. If you want the hardware reason long contexts cost so much, the KV cache article covers it.

Termination: the single hardest part of loop engineering

If there is one thing loop engineering is about, it's this: knowing when to stop. The naive loop — "let the agent decide when it's done" — is the number-one way to burn a token budget, because a confused agent will keep calling tools forever, each call looking locally reasonable. Every production loop needs explicit, external stopping conditions, and the doctrine is to have several independent exits, because any single one can fail.

The standard set of exits, and why each exists:

• A verifier (the good exit). The loop ends when an objective, automated check confirms the goal — tests pass, the schema validates, the linter is clean, the output matches the spec. Verification is the reinforcement-learning-style reward signal the loop climbs toward. Crucially, prefer a verifier over the agent's self-assessment: "I believe the task is complete" is unreliable; "the test suite is green" is not.
• Max iterations (the essential cap). A hard limit on loop turns — typical production values are 15–25 steps. This is the single most important safety control; it guarantees termination regardless of everything else.
• Wall-clock timeout. An absolute time budget (often ~300 seconds) that catches the case where individual steps are slow even if the iteration count is low — a hung tool, a slow API.
• Token / cost budget. A hard spending ceiling per run, so a loop can't quietly rack up a huge bill. Especially important given tool responses dominate tokens.
• No-progress detection (the subtle one). Fingerprint the state each turn — the tool call, its result, the file hash. If the agent produces the same state (e.g. the same failing command, identical file contents) for ~3 consecutive iterations, it's stuck in a cycle; trip a circuit breaker, stop, and alert a human. This catches the "confidently spinning" failure that the other exits miss until the budget runs out.

Error handling: turn failures into observations

In a normal program an exception unwinds the stack. In an agent loop that's usually wrong — a failed tool call is information the agent can use. The loop-engineering pattern is to catch tool errors and feed them back as observations, so the model sees "command failed: file not found" and self-corrects on the next turn, exactly as it would react to any other tool result.

try:
    result = execute(call)                    # run the tool
except ToolError as e:
    result = f"ERROR: {e}"                     # DON'T crash the loop —
                                              # hand the error to the model as an observation
messages.append({"role": "tool", "tool_call_id": call.id, "content": str(result)})

But not every error should feed the loop. Distinguish retryable from fatal:

Class	Examples	Handling
Retryable (transient)	HTTP 429, 500, 502, 503, 504; timeouts	Exponential backoff with jitter; retry a few times before surfacing to the model.
Fatal (won't fix itself)	HTTP 401, 403, 422; bad credentials; malformed request	Stop immediately — retrying wastes budget and the agent can't fix an auth failure.
Agent-correctable	file not found, test failed, wrong argument	Feed back as an observation so the model adjusts.

Loopmaxxing: the defining antipattern

The failure mode that gave loop engineering its cautionary vocabulary is loopmaxxing — the belief that if you just run an agent through enough iterations, it will eventually converge on a correct answer. Sometimes it does. Often it doesn't, and here's when it reliably fails:

• No objective success signal. If the goal is subjective ("make this essay better") with no binary pass/fail check, the loop has nothing to converge to — it will iterate indefinitely, each pass neither clearly right nor clearly wrong, until the budget dies.
• Goal drift. With an ambiguous spec, the agent's interpretation wanders across iterations; more loops make it drift further, not closer.
• Error propagation. An early mistake becomes context the agent trusts, and later steps build on it — compounding rather than correcting.
• Context overflow. Long runs fill the window with dead ends, degrading the very reasoning that's supposed to fix the problem.

The cure isn't "more loops," it's better loop design: give the loop a verifiable target so iteration has something to hill-climb, cap it so a non-converging run dies cheaply, detect stalls, and — the deeper fix — convert the parts that don't need an LLM into deterministic code. If a step is the same every time (format this, run these tests, apply this migration), take it out of the probabilistic loop and make it a plain function. The best loop engineers shrink the LLM's role to only the genuinely open-ended decisions and let deterministic scaffolding do the rest.

Andrew Ng's framing: loops inside loops

Everything above is the innermost loop — one agent iterating on one task. Andrew Ng's contribution is to zoom out and point at the nested loops that build a real product, each operating on a different timescale. His framing (paraphrased):

• The agentic coding loop (minutes). Given a spec and, ideally, a set of evals, an agent writes code, tests it, and iterates until it's bug-free and meets the spec. This is the innermost loop scaled to a whole feature — and it only works well when there's an objective verifier (the evals) to hill-climb, exactly the termination doctrine above.
• The developer feedback loop (hours). A human reviews, asks a few colleagues, ships to alpha testers, wires up A/B tests. This loop injects human context the agent doesn't have — Ng frames the human's edge as a "context advantage," not mystical "taste." Human-in-the-loop exists to feed in what the model can't know.
• The external feedback loop (days–weeks). Real users, real telemetry, real market signal. Slow, but it's the only loop that tells you whether you built the right thing at all.

Some practitioners add a fourth, outermost loop — hill-climbing / harness optimization — where you analyze production traces and automatically rewrite the loop itself (its prompts, tools, and stopping rules). That's loop engineering turned on loop engineering: the system tuning its own harness. The unifying idea across all four levels: fast inner loops need objective verifiers; slow outer loops inject information the inner loops structurally lack. Human oversight is a first-class primitive at every level, not just the top.

Verification and evals: the loop's reward signal

Since a loop is only as good as its exit, verification deserves its own treatment. A well-engineered loop has a grader that scores the agent's output against criteria. Two flavors:

• Deterministic checks. Tests pass/fail, schema validation, a linter, a regex, "does the JSON parse," "does the build succeed." Cheap, fast, unambiguous — always prefer these when the goal admits them. They're the gold-standard exit.
• LLM-as-judge. For fuzzier goals (tone, helpfulness, faithfulness), a separate model grades the output against a rubric. Necessary for subjective tasks, but noisier — treat its score as a signal, not a certainty, and calibrate it against human labels.

The engineering move is to wire the grader into the loop: a failed grade doesn't end the run, it produces feedback that goes back to the agent for another attempt (bounded by the caps). This is the difference between a loop that iterates blindly and one that iterates toward a measured target — the whole reason iteration helps at all. Investing in evals is investing in loop quality, because evals are the thing the loop optimizes.

The research lineage: from ReAct to modern loops

Loop engineering didn't appear from nowhere; it's the productionization of a research arc. The milestones and what each added:

Method	What it added to the loop	Reported result
ReAct (2022)	interleave reasoning + acting; observations feed next step	~34% gain on ALFWorld vs act-only
Reflexion (2023)	self-reflection: the agent writes a critique of its failure into memory and retries	strong gains on trial-and-error tasks
LLMCompiler (2024)	plan a DAG of tool calls and run independent ones in parallel	~3.6× speedup over sequential ReAct
Multi-agent orchestration	an orchestrator spawns worker sub-agents with isolated context	large gains on broad research tasks (Anthropic)

The trajectory is clear: start with a single reason-act loop (ReAct), add memory of failures so iteration learns (Reflexion), add parallelism so independent steps don't serialize (LLMCompiler), and add isolation so big sub-tasks don't pollute the main context (multi-agent). Each is a loop-engineering refinement, and modern harnesses combine them.

Memory: what the loop remembers between turns and runs

Context management handles a single run's window; memory handles what persists across turns and runs. Production agents use several kinds:

• Episodic — records of prior actions and their outcomes ("last time I ran this migration it failed on the index"). Lets the loop avoid repeating mistakes.
• Semantic — structured, curated domain knowledge. The CLAUDE.md pattern — a human-written project memory file — is a leading example, and it's notably more reliable than auto-generated memory because a human vetted it.
• Vector — similarity retrieval over a large corpus, pulled into context on demand (RAG inside the loop).
• File-based — state written to disk and re-read each iteration. This is the trick behind "reset the context but keep the progress" loops.

The Ralph loop: reset context, keep progress

A striking file-based pattern (nicknamed the "Ralph loop") runs an agent in an infinite shell loop that resets the model's context between iterations, reading the current state from disk each time and writing progress back. It attacks both context overflow (each iteration starts fresh, so the window never rots) and premature exit (a stop hook checks objective completion criteria before letting the loop end). It's a vivid illustration that "the loop" can live outside the model entirely — the durable state is on the filesystem, and the LLM is a stateless step invoked over and over against it. Loop engineering at its most literal: the loop is a shell script; the model is a subroutine.

Loop engineering vs prompt / context / harness engineering

The terms get muddled. A clean separation:

Discipline	Question it answers	Artifact
Prompt engineering	What do I say to the model this turn?	a prompt string / template
Context engineering	What information is in the window right now?	retrieval + assembly logic
Loop engineering	How is the model invoked repeatedly, and when does it stop?	the control loop + exits + verifier
Harness engineering	What tools, sandboxes, and guardrails surround the loop?	the tool layer + sandbox + policies

They nest: harness engineering builds the environment, loop engineering drives the cycle inside it, context engineering fills each turn's window, prompt engineering words each message. Skill in the outer layers increasingly dominates — a great prompt inside a badly-designed loop still fails, but a mediocre prompt inside a well-designed loop (good verifier, good caps, good context management) often succeeds. That inversion is exactly why the field's attention moved outward from prompts to loops.

A production rollout playbook

You don't ship a fully autonomous loop on day one. The practical progression trades autonomy for safety and earns it back with evidence:

The through-line: start with maximum human oversight and minimum autonomy, then trade oversight for verifiers and guardrails as you accumulate evidence. Autonomy is something a loop earns by demonstrating it's safe and convergent, not a default you switch on.

A minimal but real loop, end to end

Putting the pieces together — the skeleton, the exits, error-as-observation, and a verifier:

def run_agent(goal, tools, verifier, max_iters=20, budget_usd=1.0, deadline_s=300):
    messages = [SYSTEM, {"role": "user", "content": goal}]
    start, spent, recent = time.time(), 0.0, deque(maxlen=3)

    for step in range(max_iters):                       # (1) hard iteration cap
        if time.time() - start > deadline_s: return stop("timeout")   # (2) wall-clock
        if spent > budget_usd:            return stop("budget")       # (3) cost cap

        resp = call_llm(messages, tools); spent += cost(resp)
        messages.append(resp)

        if not resp.tool_calls:                          # model thinks it's done…
            if verifier(state()): return done(resp)      # (4) …but VERIFY objectively
            messages.append(nudge("verification failed, keep going"))
            continue

        for call in resp.tool_calls:
            try:    out = execute(call)
            except ToolError as e: out = f"ERROR: {e}"   # error -> observation
            messages.append(tool_msg(call.id, out))

        fp = fingerprint(state())                        # (5) no-progress detection
        recent.append(fp)
        if len(recent) == 3 and len(set(recent)) == 1:
            return stop("no progress — circuit breaker")

    return stop("max iterations")

Every line that isn't the model call is loop engineering: the caps, the verifier gate on the "done" signal, the error absorption, the stall fingerprint. The model is one call in the middle; the reliability lives in everything around it.

FAQ

Is "loop engineering" actually new, or just a rebrand of agent design?

The mechanics (ReAct-style loops) aren't new — what's new is treating the loop's design as the primary engineering surface, above prompts. The term crystallized in 2026 because models got reliable enough to run long autonomous loops and coding agents made loops the daily interface, so the loop became where reliability is won or lost. Call it a useful name for an emphasis shift rather than a brand-new technique.

What's the single most important thing to get right?

Termination. Specifically: an objective verifier for the success exit, plus independent safety caps (iterations, time, cost) and no-progress detection. Everything else improves an agent; missing exits makes it dangerous. "Let the agent decide when it's done" is the most expensive default in the field.

How many iterations should I allow?

Production loops commonly cap at 15–25 steps, with a wall-clock timeout around 300 seconds and a hard token/cost budget as backstops. The right number depends on your task, but the cap should exist regardless — it's the guarantee that the loop terminates no matter what the other conditions do.

How do I stop an agent from looping on the same failing action?

No-progress detection: fingerprint the state each turn (tool call + result + relevant file hashes) and trip a circuit breaker if it repeats ~3 times. Also add per-tool call limits — the "agent called a broken tool 400 times" incident is exactly this failure without a breaker. And feed tool errors back as observations so the model can actually adjust rather than blindly retry.

Why is my agent so expensive?

Because a loop re-sends its growing context every turn, a single agent runs ~4× the tokens of a plain chat, and multi-agent ~15×. Tool responses are ~two-thirds of the tokens, so the fix is controlling what tools return (paginate, summarize, surgical diffs) and compacting context around 80% utilization — not trimming the system prompt.

When should I use multiple agents instead of one loop?

When a task has large, separable sub-tasks whose context would pollute the main loop. An orchestrator spawns worker sub-agents with isolated context that return short summaries — keeping each loop's window clean. It costs far more tokens, so reserve it for genuinely broad tasks (deep research, large refactors) where the isolation is worth the price.

Where do humans fit in an autonomous loop?

At the exits you don't yet trust and at the information the model lacks. Early on, a human approves each action; as verifiers prove out, humans move to reviewing circuit-breaker alerts and injecting "context advantage" the agent can't obtain — domain knowledge, product judgment, real-user signal. Human-in-the-loop is a design primitive at every loop level, not a fallback.

Takeaways

• An agent is a loop: reason → act → observe → update, repeated. Loop engineering is designing that loop, especially its exit.
• The field moved prompt → context → loop → harness engineering; skill in the outer layers now dominates.
• Manage the message history: correct tool_call_id ordering, tool results are ~2/3 of tokens, compact around 80% utilization, paginate reads, prefer surgical edits.
• Termination is the hard part: an objective verifier for success, plus max-iterations (15–25), wall-clock (~300s), cost budget, and no-progress detection — several independent exits.
• Turn errors into observations (retryable vs fatal); use idempotency keys and per-tool caps so retries don't cause damage.
• Avoid loopmaxxing: more iterations don't fix a loop with no objective target — give it a verifier, cap it, and move deterministic steps out of the LLM.
• Ng's nested loops: fast agent/coding loops (minutes) inside developer (hours) and external (days) feedback loops — fast loops need verifiers, slow loops inject missing context.
• Roll out in phases: human approval → automated verifiers → circuit breakers → deterministic-ize the repetitive. Autonomy is earned.

References & further reading

• Andrew Ng — on "loop engineering" as a buzzphrase — the tweet that named the moment.
• Andrew Ng's 3 Loops for 0-to-1 AI Products — the agentic-coding / developer / external feedback loops.
• LangChain — The Art of Loop Engineering — the loop primitive, verification middleware, and the fourth (optimization) loop.
• Steve Kinney — The Anatomy of an Agent Loop — message ordering, the 67.6% token figure, stopping conditions, error classes.
• Data Science Dojo — Agentic Loops: From ReAct to Loop Engineering — the five-stage anatomy, memory types, and the research progression.
• TechTalks — Demystifying loop engineering & avoiding loopmaxxing — the antipattern and the phased rollout.
• Yao et al. — ReAct: Synergizing Reasoning and Acting in Language Models — the foundational loop pattern.
• Shinn et al. — Reflexion — self-reflection and verbal reinforcement across iterations.