Chapter Objective: Build a precise mental model of the Agent Loop—the fundamental execution cycle that drives tool-using AI Agents. The loop itself is remarkably small. The real engineering challenge lies in the runtime mechanisms that surround it: tool execution, state management, error recovery, context management, termination control, verification, and long-running execution.
3.1 The Minimal Agent Loop
If an AI Agent has a minimal computational core, it can be reduced to a surprisingly small execution loop:
while not finished:
response = model(context)
if response.tool_call:
result = execute_tool(response.tool_call)
context.append(result)
else:
return response
Conceptually, this loop performs three operations:
- Generate — ask the model what to do next.
- Act — execute the requested tool invocation.
- Observe — feed the resulting information back into the model’s working context.
The loop then repeats until a termination condition is reached.
This is enough to describe the fundamental behavior of a tool-using Agent:
The model selects the next action, the environment executes it, and the resulting observation becomes input to the next model invocation.
The implementation above is intentionally simplified. A production Agent runtime must additionally deal with:
- multiple tool calls in a single model response,
- tool-call/result message pairing,
- streaming responses,
- model failures,
- tool failures,
- retries,
- context limits,
- cancellation,
- permission checks,
- termination conditions,
- state persistence,
- and resource budgets.
Consequently, the conceptual loop may fit into fewer than ten lines of code, while a production implementation can grow to hundreds or thousands of lines.
This distinction is fundamental:
The Agent Loop is simple by design. The complexity is implemented around the loop.
Tool Calls and Tool Results Must Remain Associated
There is an important implementation detail hidden by the simplified example.
When a model generates a tool call, the corresponding tool result must be associated with that invocation when the conversation state is updated.
Conceptually:
Model message
↓
tool_use
↓
Tool execution
↓
tool_result
↓
Next model invocation
Appending only the tool result is insufficient because the model also needs to know which tool invocation produced that result.
Modern model APIs therefore generally represent tool calls and their results as structured messages or blocks with explicit relationships between them.
The minimal example hides this protocol detail because the goal is to illustrate the control flow rather than the complete message protocol.
3.2 The Agent Loop in a Real Agent Runtime
A production Agent runtime follows the same fundamental pattern, but introduces state, streaming, termination, and recovery logic.
A simplified representation of the loop can be expressed as:
async function* queryLoop(params: QueryParams) {
let state = {
messages,
toolUseContext,
turnCount: 1,
transition: undefined
};
while (true) {
// Generate a model response
const response =
yield* streamModelResponse(state.messages);
// Evaluate whether execution should terminate
if (response.terminal) {
return response.terminal;
}
// Execute tool calls produced by the model
const results =
yield* runTools(response.toolCalls);
// Feed tool results back into the conversation state
state.messages.push(...results);
state.turnCount++;
}
}
This simplified structure exposes several important architectural properties.
1. The loop is potentially unbounded
The loop does not necessarily know in advance how many iterations a task will require.
Instead, execution continues until an internal termination condition is satisfied.
This is fundamentally different from a traditional deterministic workflow, where the number and ordering of steps are usually known beforehand.
2. Each turn follows a Generate–Act–Observe pattern
At a high level:
Generate
↓
Act
↓
Observe
↓
Generate
↓
Act
↓
Observe
↓
...
The exact runtime implementation may combine or reorder some of these operations, but the feedback-driven structure remains.
3. State persists across turns
The Agent cannot treat each model invocation as an isolated request.
It needs to preserve information such as:
- conversation messages,
- tool calls and results,
- task progress,
- execution metadata,
- permissions,
- model state,
- runtime transitions,
- and other session-level information.
The state object is therefore more than a convenience. It represents the beginning of Agent state management.
4. The Loop is primarily an orchestration mechanism
The loop itself does not need to understand how the model reasons, how a file system works, or how a database query is executed.
Instead, it coordinates specialized subsystems:
Agent Loop
│
┌────────────┼────────────┐
↓ ↓ ↓
Model Tools Context
│ │ │
Reasoning Execution Working State
This separation of responsibilities is one of the most important architectural properties of modern Agent systems.
3.3 Observe → Think → Act
The Agent Loop can be described at a conceptual level as:
Observe → Think → Act
This model is related to the classical perception–reasoning–action paradigm in AI and to feedback-loop concepts from control theory.
For an LLM-based Agent, the abstraction can be expressed as:
Observe
Read the available information:
user request, conversation history,
tool results, files, environment state
↓
Think
Determine what the available information means
and what action should be taken next
↓
Act
Call a tool or produce a response
↓
Observe
The result of the action becomes new information
↓
Repeat
The key idea is not the existence of three isolated stages.
It is the continuous feedback relationship between them.
A conventional chatbot can often be modeled as:
Input → Model → Output
An Agent introduces an external environment and therefore becomes:
Input
↓
Model
↓
Action
↓
Environment
↓
Observation
↓
Model
↓
Action
↓
...
This transforms LLM generation from a one-shot interaction into an iterative decision-and-action process.
Example: Fixing a Software Defect
Consider an Agent asked to fix:
TypeError: foo is not a function
A simplified execution trace might look like:
Observe:
The user reports a runtime error involving foo.
↓
Think:
The error may be caused by an incorrect definition,
import, or invocation of foo.
↓
Act:
Search the codebase for foo.
↓
Observe:
foo is defined in src/utils.ts.
↓
Think:
Inspect the implementation and its callers.
↓
Act:
Read src/utils.ts and the relevant calling code.
↓
Observe:
foo is a function, but one caller treats it as an object.
↓
Think:
The caller should invoke foo as a function.
↓
Act:
Modify the calling code.
↓
Act:
Run the relevant test suite.
↓
Observe:
Tests pass.
↓
Act:
Report the completed fix.
The important point is that the Agent does not need to know the entire execution path before it starts.
Each observation changes the information available for the next decision.
That is what makes the architecture adaptive.
“Think” Is a Conceptual Abstraction
There is an important distinction between the conceptual term Think and the model’s internal reasoning process.
Think should not be interpreted as requiring a separately exposed chain-of-thought phase.
A modern model may produce a tool call directly from its generation process:
Model generation
↓
Structured tool call
↓
Tool execution
The model may internally perform substantial reasoning without exposing that reasoning to the application or user.
Therefore:
Observe → Think → Act is an architectural abstraction, not a requirement that an Agent expose its internal reasoning process.
This distinction becomes particularly important when designing production Agents, because observability should focus on useful operational signals—such as decisions, tool calls, outcomes, and state transitions—rather than assuming that internal reasoning must be exposed.
3.4 ReAct
ReAct, short for Reasoning and Acting, was introduced by Yao et al. in 2022 and became one of the influential approaches in early LLM Agent research.
Its central idea was to combine reasoning and interaction with the environment in an iterative process.
A simplified ReAct trace looks like:
Thought:
I need the current weather before answering the user.
Action:
get_weather({"city": "Beijing"})
Observation:
{"weather": "rain", "temperature": 18}
Thought:
It is raining, so the user should bring an umbrella.
Action:
Finish("Bring an umbrella today.")
The classic ReAct pattern contains three conceptual elements:
- Thought — reasoning about the current state.
- Action — interaction with an external tool or environment.
- Observation — information returned by that interaction.
Its major contribution was demonstrating the value of interleaving reasoning and environment interaction rather than treating reasoning as an entirely isolated preprocessing step.
From ReAct Text to Native Tool Calling
Early implementations often represented actions as text:
Action: get_weather({"city": "Beijing"})
The application then had to parse that text and convert it into an executable operation.
This approach introduces an unnecessary interface boundary:
LLM
↓
Generated text
↓
Parser
↓
Tool invocation
Native Tool Calling provides a more robust representation:
LLM
↓
Structured tool call
↓
Tool executor
Instead of asking the model to produce a textual command that the application must interpret, the model generates a structured invocation containing the tool name and arguments.
This reduces parsing ambiguity and allows the runtime to validate tool names and parameters before execution.
The relationship between ReAct and modern Tool Calling should therefore be understood carefully:
Native Tool Calling is not simply “ReAct implemented differently.” Rather, it provides a structured runtime mechanism for implementing the broader reasoning–action interaction pattern that ReAct helped popularize.
The explicit Thought / Action / Observation text format has largely disappeared from production implementations, but the underlying iterative interaction model remains highly relevant.
3.5 Plan-and-Execute
Plan-and-Execute takes a different approach.
Instead of deciding only what to do next, the Agent first constructs a higher-level plan and then executes that plan.
For example:
Plan:
1. Inspect the project structure
2. Locate the relevant implementation
3. Analyze the defect
4. Design the modification
5. Apply the change
6. Run tests
7. Verify the result
Execution then proceeds against this plan.
Strengths
Global task awareness
Planning provides the Agent with a higher-level representation of the task.
Predictability
The plan can be inspected before execution begins.
Suitable for structured tasks
Plan-and-Execute works well when:
- the task has relatively clear dependencies,
- the environment is stable,
- and the major execution steps can be predicted in advance.
Limitations
Plans can become stale
Execution generates new information.
That information may invalidate the original plan.
For example:
Initial assumption:
The bug is in module A.
Execution:
Module A is correct.
The problem actually originates in module C.
Original plan:
No longer valid.
Planning introduces additional cost
The Agent must spend model capacity and latency constructing the plan before performing useful work.
Planning can reduce adaptability
Highly exploratory tasks, such as debugging unfamiliar systems, often cannot be planned accurately in advance.
Dynamic Planning Is More Useful Than Static Planning
This leads to an important distinction:
Static Plan-and-Execute
Plan
↓
Execute
↓
Finish
versus:
Dynamic Planning
Plan
↓
Execute
↓
Observe
↓
Update Plan
↓
Execute
↓
Observe
↓
...
Modern coding Agents often use the second approach.
A task list can become a persistent representation of execution state rather than a document generated once and discarded.
This is an important architectural evolution:
Planning can be treated as dynamic state rather than a one-time precondition for execution.
In Claude Code–style workflows, mechanisms such as task lists and explicit planning modes provide this capability without forcing every task into a rigid Plan-and-Execute pipeline.
3.6 Tool-use Agents
Tool-use Agents represent the dominant execution pattern in many modern LLM Agent systems.
Their defining characteristic is:
The model selects the next action based on the current state rather than generating a complete execution plan upfront.
The conceptual loop is:
while not finished:
response = model(context)
if response.tool_call:
result = execute_tool(response.tool_call)
context.append(result)
else:
return response
The key property is incremental decision-making.
The Agent does not need to know the complete solution before interacting with the environment.
Instead:
Current State
↓
Next Decision
↓
Action
↓
New State
↓
Next Decision
This makes Tool-use Agents particularly effective for open-ended tasks such as:
- debugging,
- code refactoring,
- repository exploration,
- incident investigation,
- research,
- data analysis,
- and system administration.
Adaptability Comes at a Cost
The same flexibility that makes Tool-use Agents powerful also introduces uncertainty.
An Agent may:
- explore irrelevant information,
- repeat actions,
- take inefficient paths,
- misinterpret tool results,
- or fail to recognize when the task is complete.
These are not necessarily failures of the LLM alone.
They are runtime problems.
They require mechanisms such as:
- termination policies,
- budgets,
- permissions,
- verification,
- state tracking,
- error recovery,
- context management,
- and observability.
In other words:
Tool-use provides the Agent with flexibility; the Harness provides the constraints that make that flexibility operationally safe.
Why Tool-use Agents Work Well for Open-ended Tasks
Many real-world engineering tasks are not fully specified in advance.
For example:
“Find out why this service is intermittently failing.”
Before investigation begins, the Agent cannot know:
- which component is responsible,
- which logs matter,
- whether the failure is reproducible,
- which dependencies are involved,
- or what evidence will be discovered.
A rigid plan is therefore likely to become obsolete.
Tool-use execution is better suited to this environment because the Agent can continuously incorporate new evidence.
By contrast, deterministic processes with known steps are often better implemented as traditional Workflows rather than Agents.
This gives us a useful architectural rule:
Use Workflows when the process is known. Use Agents when the path depends on information discovered during execution.
3.7 Reflection
Reflection introduces an additional feedback mechanism in which an Agent evaluates an intermediate result or its own approach before continuing.
A simplified Reflection loop is:
Generate
↓
Evaluate / Critique
↓
Revise
↓
Generate Again
For example:
Generate:
Produce a proposed solution.
Critique:
Identify potential errors, omissions,
or unsupported assumptions.
Revise:
Modify the solution based on the evaluation.
Reflection can improve performance on tasks where the quality of the initial output is difficult to guarantee.
However, Reflection is not automatically beneficial.
Each additional evaluation cycle introduces:
- additional model calls,
- additional latency,
- additional token consumption,
- and potentially additional failure modes.
Therefore, Reflection should be treated as an engineering trade-off, not as a universally beneficial Agent capability.
Reflection in Coding Agents
In software engineering, Reflection often emerges naturally from external feedback.
For example:
Edit
↓
Run Test
↓
Test Failure
↓
Analyze Failure
↓
Modify Code
↓
Run Test Again
A separate “reflection stage” is not necessarily required.
The test result itself provides an evaluation signal that drives the next iteration.
This is an important distinction:
External verification can provide a stronger reflection signal than self-evaluation alone.
A model evaluating its own answer may still share the same blind spots that produced the original answer.
An independent test suite, compiler, type checker, or validator provides a different source of evidence.
3.8 Self-correction
Self-correction refers to the Agent’s ability to detect that an action or intermediate result is incorrect and adapt accordingly.
A coding Agent provides a particularly clear example:
Read
↓
Understand
↓
Modify
↓
Test
↓
Observe Failure
↓
Analyze
↓
Fix
↓
Test Again
↓
Pass
The important property is not simply that the model can “correct itself.”
It is that the environment provides a feedback signal that allows the Agent to determine whether its previous action was successful.
Software engineering is unusually well suited to this pattern because it provides many objective signals:
- compiler errors,
- type-checking failures,
- unit-test failures,
- integration-test failures,
- linter warnings,
- runtime exceptions,
- static-analysis findings.
These signals are generally:
- immediate,
- machine-generated,
- relatively objective,
- and directly related to the action that produced them.
Why Coding Agents Have an Advantage
Consider two tasks.
Task A:
“Fix this TypeScript compilation error.”
The compiler provides an objective evaluation signal.
Task B:
“Create the best possible vacation itinerary.”
The quality of the result is much more subjective.
There may be constraints and measurable factors, but there is no universal equivalent of a compiler that can declare:
PASS
or:
FAIL
This distinction helps explain why coding has become one of the strongest application domains for Agents.
The important factor is not simply that code is structured.
It is that software development provides a rich, automated feedback environment.
This leads to a broader principle:
Agent reliability depends heavily on the quality of the feedback signals available during execution.
The better the environment can evaluate an Agent’s actions, the more effectively the Agent can iterate and self-correct.
3.9 Why Is the Agent Loop So Simple?
We can now return to the minimal loop:
while not finished:
response = model(context)
if response.tool_call:
result = execute_tool(response.tool_call)
context.append(result)
else:
return response
Why can such a small piece of code support such complex behavior?
Because the complexity has not disappeared.
It has been distributed across specialized subsystems.
The Model
model(context)
The model provides capabilities such as:
- interpreting the task,
- reasoning about available information,
- selecting tools,
- generating tool arguments,
- interpreting tool results,
- and determining the next action.
The Tool System
execute_tool(tool_call)
The Tool System provides the operational capabilities:
- file access,
- command execution,
- database queries,
- API calls,
- code modification,
- external service interaction.
Context Management
context
The context provides the working state required by the model:
- conversation history,
- tool calls,
- tool results,
- instructions,
- relevant files,
- intermediate information,
- and other state.
The Agent Loop therefore performs primarily coordination, while specialized components perform the difficult work.
This leads to one of the most important architectural insights in this book:
The simplicity of the Agent Loop is a consequence of abstraction, not evidence that Agent systems are simple.
The complexity has simply moved into:
Model
Tool System
Context Management
Runtime / Harness
3.10 The Real Complexity Is Around the Loop
A production Agent is much more than its loop.
The core execution cycle may be relatively small, while the surrounding runtime can contain a substantial amount of infrastructure.
A production Agent typically needs to answer questions such as:
| Runtime Question | Typical Harness Mechanism |
|---|---|
| How do we prevent runaway execution? | Stop conditions, turn limits, budgets |
| What happens when a tool fails? | Tool error handling and recovery |
| What happens when the model fails? | Retry, backoff, failover |
| What happens when context becomes too large? | Compaction and context management |
| How do we know the task is complete? | Verification and completion criteria |
| How do we track progress? | Task state and progress tracking |
| What happens after a crash? | Persistence and checkpointing |
| How do we control side effects? | Permissions and authorization |
| How do we understand what happened? | Logging and observability |
| How do we control cost? | Token, time, and resource budgets |
This is the fundamental reason an Agent Harness can be dramatically larger than the Agent Loop itself.
The Loop answers:
“How do we continue execution?”
The Harness answers:
“How do we make that execution safe, reliable, observable, recoverable, and controllable?”
That distinction is central to understanding Agent engineering.
3.11 Preventing Runaway Execution
One of the most fundamental Agent runtime problems is runaway execution.
An Agent may repeatedly perform actions without making meaningful progress:
get_data
↓
Incomplete result
↓
get_data
↓
Incomplete result
↓
get_data
↓
...
Or:
search
↓
read result
↓
search again
↓
read another result
↓
search again
↓
...
Turn Limits
The simplest safeguard is a maximum number of turns:
max_turns
This provides a hard upper bound on execution.
However, a turn limit is a blunt instrument.
A productive task might legitimately require many iterations, while an unproductive task might become stuck after only a few.
Therefore:
Turn limits are a safety boundary, not a complete termination strategy.
Diminishing Returns
A more sophisticated runtime can monitor whether additional iterations are producing meaningful progress.
Conceptually:
High information gain
↓
Continue
Moderate information gain
↓
Continue cautiously
Low information gain
↓
Consider termination
Repeated low information gain
↓
Terminate or escalate
For example:
Round 1:
Discover 50 relevant files.
High information gain.
Round 2:
Inspect the key implementation.
High information gain.
Round 3:
Modify the code.
Moderate information gain.
Round 4:
Run tests.
Useful validation signal.
Round 5:
Read the same file again.
Low information gain.
Round 6:
Repeat the same investigation.
Very low information gain.
The exact implementation and thresholds are runtime-specific.
The architectural principle is more general:
Agent termination can be based not only on elapsed turns, but also on evidence of meaningful progress.
This is analogous to monitoring marginal utility in an optimization process.
When additional computation produces little new information, continuing indefinitely is unlikely to improve the outcome.
3.12 Tool Errors
Tool failures are normal in real Agent environments.
Examples include:
- file not found,
- invalid command arguments,
- process failure,
- network timeout,
- permission denial,
- API errors,
- malformed tool parameters.
A robust Agent should not necessarily terminate when a tool fails.
Instead, the error can become part of the Agent’s observation:
try:
result = execute_tool(tool_call)
except ToolError as e:
result = {
"is_error": True,
"content": str(e)
}
context.append(result)
The model can then decide how to respond.
For example:
Tool:
FileNotFoundError: src/config.ts
Model:
The expected file does not exist.
I should inspect the repository structure
before attempting another path.
This transforms:
Tool Failure → Agent Failure
into:
Tool Failure
↓
Observation
↓
Model Re-evaluation
↓
Alternative Action
Retryable vs. Non-retryable Errors
A mature runtime should distinguish between error categories.
For example:
| Error | Typical Response |
|---|---|
| Network timeout | Retry |
| Temporary provider error | Retry with backoff |
| Rate limit | Wait and retry |
| File not found | Re-evaluate path |
| Invalid parameter | Correct invocation |
| Permission denied | Request authorization or stop |
| Invalid credentials | Report and terminate |
This distinction is critical.
Blindly retrying every error can create another failure mode: retry storms.
The Architectural Principle
Returning an error as an observation preserves the Agent Loop and allows the model to adapt.
This illustrates a broader principle:
Let the model handle uncertainty; let the Harness enforce deterministic safety and operational constraints.
The model can decide what to do about a missing file.
The Harness should determine whether it is allowed to execute a destructive command.
These are different responsibilities.
3.13 Model Errors
Model invocation can fail independently of tool execution.
Typical causes include:
- network failures,
- rate limiting,
- provider overload,
- authentication errors,
- malformed responses,
- service unavailability,
- output truncation.
These failures occur at the model boundary:
response = model(context)
The appropriate response depends on the error category.
Retry
Transient failures can often be retried using:
Exponential backoff with jitter
Failover
A production system may route the request to another model or provider if the primary model becomes unavailable.
Conceptually:
Primary Model
↓
Failure
↓
Retry
↓
Failure persists
↓
Fallback Model
Abort
Non-recoverable failures should terminate the execution gracefully.
For example:
Invalid credentials
Unsupported model
Invalid request
Policy rejection
Output Truncation
An interesting special case occurs when the model reaches its maximum output limit.
The Agent may receive an incomplete response even though the task is not complete.
A runtime may attempt to recover by continuing generation from the existing state:
Output truncated
↓
Preserve state
↓
Continue generation
↓
Evaluate result
If continuation repeatedly fails to make progress, the runtime can escalate or terminate.
This illustrates an important reliability principle:
Recovery should be progressive rather than binary.
Instead of:
Success / Failure
a mature runtime often implements:
Success
↓
Retry
↓
Recover
↓
Failover / Escalate
↓
Terminate
3.14 Context Overflow and Compaction
Long-running Agents continuously accumulate context:
Conversation
+ Tool Calls
+ Tool Results
+ File Contents
+ Intermediate State
+ Execution Logs
+ Model Responses
Eventually, the context approaches the model’s effective context limit.
This creates the Context Overflow problem.
The primary solution is Context Compaction.
Instead of retaining every historical token, the runtime transforms historical information into a more compact representation.
Conceptually:
Large History
↓
Summarize / Compress
↓
Compact State
↓
Continue Execution
Compaction can take several forms.
Automatic Compaction
Trigger compaction when the context approaches a configured threshold.
Reactive Compaction
Trigger compaction after a context-length failure.
Incremental Compaction
Compact smaller portions of state continuously rather than performing a large operation infrequently.
Selective Trimming
Remove information considered less relevant to the current task.
Context Replacement
Replace large historical sections with a compact summary or derived state.
The exact mechanisms vary by Agent runtime.
Source-code caveat: The March 31, 2026 leaked Claude Code source is incomplete. Some compaction-related identifiers appear through dynamic loading references without corresponding source files in the leaked tree. Their exact implementation and source locations should therefore be treated as unverified unless independently confirmed.
Compaction Is an Optimization Problem
Compaction is not free.
Generating a summary may itself require model computation, tokens, and latency.
Therefore:
Compact too early
→ unnecessary cost
Compact too late
→ context overflow risk
Compact too aggressively
→ important information may be lost
Compact too conservatively
→ insufficient context reduction
The goal is therefore not:
“Compress as much as possible.”
The goal is:
“Preserve the information required for future decisions while minimizing context cost.”
This makes Context Compaction one of the most important areas of Agent runtime engineering.
3.15 Premature Completion
Runaway execution has an opposite failure mode:
The Agent stops before the task is actually complete.
For example:
Agent:
"The issue has been fixed."
Reality:
The code was modified,
but the tests were never executed.
Premature completion can result from:
- ambiguous task requirements,
- unclear completion criteria,
- insufficient verification,
- conservative behavior,
- missing feedback signals.
Explicit Completion Criteria
The Agent should have a clear definition of what constitutes success.
For example:
The task is complete only when:
1. The implementation has been modified.
2. Unit tests pass.
3. Integration tests pass.
4. No new type errors are introduced.
Verification Before Completion
A robust coding Agent should distinguish:
I changed the code.
from:
The change is correct.
The second statement requires verification.
A typical pattern is:
Modify
↓
Validate
↓
Observe
↓
Correct if necessary
↓
Validate again
↓
Complete
Task Tracking
Persistent task state can also reduce premature completion:
[x] Locate defect
[x] Modify implementation
[x] Add test
[ ] Run integration tests
[ ] Verify final result
This gives the Agent an explicit representation of incomplete work.
Premature Stop and Runaway Execution Are Related
These two failure modes appear opposite:
Too early → Premature Completion
Too late → Runaway Execution
But both are manifestations of the same underlying problem:
The runtime does not have a sufficiently reliable definition of completion.
Therefore, a production Agent needs both:
- termination controls, and
- completion verification.
The first prevents the Agent from continuing forever.
The second prevents it from stopping too soon.
3.16 Long-running Agents
A Long-running Agent extends the Agent Loop from a short interactive session into a process that may operate for hours or longer.
Examples include:
- large-scale codebase refactoring,
- database migration,
- infrastructure modernization,
- large research tasks,
- automated testing and remediation.
Long-running execution amplifies almost every problem discussed above.
Runaway execution
→ wasted compute and tokens
Context overflow
→ eventually becomes unavoidable
Premature completion
→ more opportunities for incorrect completion
Transient failures
→ more opportunities for interruption
State loss
→ potentially large amounts of work can be lost
A production Long-running Agent therefore needs additional infrastructure.
Checkpointing
Persist meaningful execution state periodically:
Task
↓
Checkpoint
↓
Continue
↓
Checkpoint
↓
Continue
If execution fails, the Agent can resume from a known state.
Task Decomposition
Large objectives should be represented as smaller units:
[x] Analyze architecture
[x] Refactor module A
[x] Refactor module B
[ ] Refactor module C
[ ] Run integration tests
[ ] Verify deployment
This provides both operational control and progress visibility.
Context Management
Long-running execution requires continuous context management.
The system must be able to preserve important information while preventing the context from growing without bound.
Resource and Cost Governance
A long-running Agent can consume substantial:
- tokens,
- compute,
- tool executions,
- API requests,
- and financial resources.
Therefore, the runtime may require:
- token budgets,
- execution time limits,
- tool quotas,
- concurrency limits,
- cost budgets,
- and cancellation mechanisms.
This leads to a broader principle:
A production Agent must know not only what it can do, but also when it should stop doing it.
3.17 The Agent Loop vs. the Agent Harness
We can now establish a clean architectural boundary.
Agent Loop
The Agent Loop is responsible for the fundamental execution cycle:
Generate
↓
Act
↓
Observe
↓
Repeat
Agent Harness
The Harness provides the infrastructure required to execute that loop reliably:
Agent Harness
│
┌───────────────────┼───────────────────┐
↓ ↓ ↓
Model Runtime Tool Runtime Context Runtime
│ │ │
├── Retry ├── Permissions ├── Compaction
├── Failover ├── Validation ├── Trimming
└── Routing └── Error Handling └── Persistence
┌───────────────────────────────────────────────┐
│ State • Observability • Checkpoints • Budgets │
│ Verification • Hooks • Cancellation │
└───────────────────────────────────────────────┘
The Loop defines how the Agent iterates.
The Harness defines the operational environment in which that iteration occurs.
This distinction is foundational to the architecture of production-grade Agents.
Chapter Summary
The Agent Loop is conceptually small, but architecturally fundamental.
The key ideas from this chapter are:
1. The Agent Loop is a feedback-driven execution cycle
At its core:
Generate
↓
Act
↓
Observe
↓
Repeat
The model determines the next action, the environment executes it, and the resulting observation becomes input to the next iteration.
2. Observe → Think → Act is a conceptual model
It describes the cognitive and environmental interaction of an Agent.
It should not be interpreted as three mandatory runtime phases or as requiring exposure of the model’s internal reasoning.
3. ReAct established an important reasoning–action pattern
ReAct demonstrated the value of interleaving reasoning with interaction.
Modern Native Tool Calling provides a structured execution mechanism that avoids the fragility of parsing textual actions.
4. Plan-and-Execute and Tool-use represent different planning strategies
Plan-and-Execute emphasizes upfront planning.
Tool-use Agents emphasize incremental decision-making.
Modern systems can combine both through dynamic planning and persistent task state.
5. Reflection and Self-correction depend on feedback
Reflection evaluates intermediate results or approaches.
Self-correction adapts behavior based on detected errors.
The strongest feedback often comes from external evaluators such as tests, compilers, validators, and runtime environments.
6. The Agent Loop is simple because complexity is delegated
The Loop relies on:
LLM
Tool System
Context Management
and coordinates them rather than implementing their internal complexity itself.
7. The real engineering challenge is the surrounding Harness
Production Agents must address:
- runaway execution,
- tool errors,
- model errors,
- context overflow,
- premature completion,
- state persistence,
- checkpointing,
- verification,
- permissions,
- observability,
- and resource governance.
This leads to the central architectural principle of this chapter:
The Agent Loop is the execution core. The Agent Harness is the engineering system that makes that core reliable, controllable, and usable in the real world.
With this foundation, we can now move into Part II — Agent Harness.
If the Agent Loop is the heartbeat, the Harness is the runtime system that keeps that heartbeat operating safely and continuously.
Related posts:
Chapter 2 — LLM Application Architecture: Core Components and Engineering Foundations
Chapter 1: Rethinking AI Agents
A Comprehensive Guide to AI Agents: Definition, Role, Examples, and Future Prospects
AI Agent Architecture and Engineering Practice Guide
The Key Milestones in the History of Artificial Intelligence (2026.08)
Building a Python-based AI Agent with LangGraph and OpenRouter: A Hands-On Guide
Clone a WordPress with ASP.NET and React Part 1: Initialize Project Structure with AI
Artificial Intelligence (AI) Learning Roadmap for Beginners in 2025