<<AI Agent Architecture Design and Engineering Practices>> Chapter 1: Rethinking AI Agents

Chapter Objective: Build an accurate understanding of AI Agents, clarify the boundaries between Chatbots, Copilots, Agents, Autonomous Agents, and Coding Agents, understand why an LLM can serve as the “brain” of an Agent, and establish the fundamental definition of an Agent: Agent = LLM + Harness. This chapter provides the conceptual foundation for the entire book.

1.1 A Brief History: How the Harness Was “Forced” Into Existence

To understand the formula Agent = LLM + Harness, it is useful to understand how Agents evolved and, along the way, how the need for a Harness emerged.

2017  Transformer introduced          → The foundation of modern LLMs was established

2020  GPT-3 scaled up                 → Reasoning and general-purpose capabilities emerged

2022  ChatGPT released                → The Chatbot era: LLMs learned to "talk"

2023  AutoGPT / BabyAGI               → The autonomous Agent wave, exposing critical limitations

2023  Function Calling became common  → Tool Calling became a standard API capability

2024  Coding Agents emerged           → Coding Agents began to mature

2024  MCP released                    → Tools moved from private implementations to a standard protocol

2025  Claude Code released            → Coding Agents entered an engineering-driven phase

2025  Production-grade Agents        → Security, observability, and evaluation became key concerns

One pattern is clear: every step added a capability while simultaneously exposing a new gap. The following sections explain this progression.

1.1.1 2017–2020: The Birth and Scaling of LLMs

In 2017, Vaswani et al. introduced the Transformer architecture in Attention Is All You Need (see Section 1.4.1), establishing the architectural foundation of modern LLMs. OpenAI then expanded the GPT family: GPT-1 (2018), GPT-2 (2019), and GPT-3 (2020). GPT-3 marked an important turning point. As model scale crossed a certain threshold, capabilities emerged that were not commonly observed in smaller models: few-shot learning, code generation, and basic logical reasoning.

The key insight at this stage is: an LLM was not designed to be an “Agent”; it was trained to predict the next token. Yet it became sufficiently general-purpose that people began to ask: could a model do more than answer questions? Could it actually do things?

1.1.2 2022: The Chatbot Era, and the Externalization of Reasoning

The release of ChatGPT at the end of 2022 brought LLMs from research labs into mainstream use. However, ChatGPT in its original form was primarily a Chatbot—it could “talk,” but it could not “act.” It could not read files, query databases, or execute actions.

At roughly the same time, two academic developments became particularly important to the later evolution of Agents:

Chain-of-Thought (CoT, Wei et al., 2022): By prompting models to “think step by step,” researchers found that complex reasoning accuracy could be significantly improved. This demonstrated that reasoning could be externally elicited and guided.
ReAct (Yao et al., 2022): ReAct interleaved Reasoning and Acting—the model reasons, invokes an external tool, observes the result, and then continues reasoning. This became an early conceptual precursor to the Agent Loop.

Together, these two developments pointed in the same direction: a carefully designed loop could enable a model that only “talked” to begin learning how to “act.”

1.1.3 2023: The Autonomous Agent Wave and the “Harness Gap”

In March 2023, AutoGPT and BabyAGI helped trigger a wave of interest in “autonomous Agents.” Their basic idea was straightforward: give an LLM a goal, provide tools, and let it run through a loop of “plan → execute → reflect → re-plan” with minimal human intervention.

The results quickly exposed major problems: infinite loops, context explosion, task drift, and uncontrolled costs. AutoGPT could spend hours circling around a goal, consuming large amounts of tokens while producing little useful output.

This failure is one of the most valuable lessons for understanding Agent architecture: LLM + loop alone is not enough to build a reliable Agent. Beyond the loop, a complete infrastructure layer is required. Who manages context? Who executes tools? Who decides whether the Agent should stop? Who enforces permissions? Who persists state?

Many of these capabilities were missing or immature in early autonomous Agents. They were not simply failed products; they demonstrated, at significant cost, that the missing layer between a “bare model + loop” and a reliable Agent is what later came to be described as the Harness.

1.1.4 2023–2025: The Maturation of Tool Calling and Coding Agents

In mid-2023, OpenAI introduced Function Calling in its API, and other model providers followed. This marked an important transition: tool use moved from the fragile practice of prompting models to emit JSON into a native model capability. Models could now be trained to emit structured tool-call requests when actions were required, rather than relying entirely on free-form text.

Once tool calling became standardized, Coding Agents began to emerge rapidly. Starting in 2024, products such as Cursor, Cline, and Aider matured quickly. In early 2025, Claude Code was released, pushing Coding Agents further toward an engineering-oriented model of operation.

These products brought Agents into real software engineering workflows: reading code, modifying code, running tests, inspecting errors, and repairing failures. Coding became an especially effective proving ground for Agents because it provides rich objective feedback signals—compiler errors, failed tests, and type-checking results. These signals make the Agent’s self-correction loop genuinely actionable, unlike many general-purpose Agent scenarios where the system can easily drift or loop without meaningful progress.

At the end of 2024, Anthropic introduced MCP (Model Context Protocol), aiming to turn tools from framework-specific implementations into a reusable standard protocol. Tool integration therefore began moving toward greater standardization and interoperability.

1.1.5 The Historical Conclusion: Agent = LLM + Harness

When the evolution above is viewed as a whole, the conclusion becomes clear:

An LLM and a loop alone cannot support a reliable, secure, and controllable Agent. What makes an Agent practical is the software system surrounding the LLM—the system that provides context, executes tool requests, drives the decision loop, establishes boundaries for actions, and records behavior. This system is the Harness.

This is the historical origin of the book’s central formula:

Agent = LLM + Harness

This section introduces the conclusion first. The formal definition and detailed decomposition will be covered in Section 1.6. Before that, we need another piece of the puzzle: the capability spectrum from Chatbot to Copilot to Agent.

1.2 From Chatbot to Agent: The Capability Spectrum

The previous section described the timeline. This section describes the capability spectrum. The timeline explains how the Harness was forced into existence; the capability spectrum explains how Chatbots, Copilots, and Agents differ.

“AI Agent” has become one of the most overused terms in the technology industry. Almost every product with some LLM capability is eager to label itself an Agent. But anyone who genuinely understands Agent architecture should be cautious about this generalization. Calling a Chatbot an Agent is not only inaccurate; it can also obscure the fundamental architectural differences between the two.

To build an accurate mental model, follow the capability progression:

Chatbot → Copilot → Agent → Autonomous Agent → Coding Agent

1.2.1 What Is a Chatbot?

A Chatbot is the most basic form of an LLM application. Its essence is an “almost stateless question-and-answer engine”:

User input → LLM generates a response → Return response

Its characteristics are clear:

Primarily single-turn: Each interaction is largely independent. Even when multiple conversation turns are supported, the history is generally included as context rather than becoming a persistent side effect.
No tools: It can “talk,” but it cannot inherently “act.” It cannot read files, query databases, or send emails without an external execution layer.
No persistent state: After the conversation ends, the system does not inherently remember things or change the state of the external world.
Passive response: It responds when prompted; it does not independently initiate actions.

Early forms of ChatGPT and many AI customer-service bots fall into this category. Their value lies in conversational information access; their limitation is that they can “talk,” but cannot independently “take action.”

1.2.2 What Is a Copilot?

A Copilot is an important evolution beyond the Chatbot. The term was popularized by GitHub Copilot, and its central metaphor is:

AI is not the person making decisions for you; it is the assistant sitting beside you and helping you work.

The architectural characteristics of a Copilot include:

Human-led, AI-assisted: The human remains the “driver,” controlling direction and final decisions. AI provides suggestions, completions, and recommendations.
Embedded in the workflow: A Copilot is deeply integrated into a specific context—GitHub Copilot is embedded in the IDE, while Microsoft Copilot is embedded in Office. It is contextual rather than purely generic.
Incremental output: It tends to complete the next token or small unit of work rather than independently completing an entire task. In coding, it may complete the next line or function; in writing, the next paragraph.
Synchronous collaboration: Each AI output is immediately presented to the human, who can review, accept, or reject it.

The importance of the Copilot model is that it demonstrated how LLMs could create value in real productivity scenarios. But it remains constrained by one key characteristic: it is passive. It does not independently decide what to do next; it responds when the human provides direction.

1.2.3 What Is an Agent?

An Agent represents a qualitative leap. Its defining characteristics are Autonomy and Tool Use:

It can use tools: An Agent does not merely “talk”; it can “act.” It can call functions, read and write files, execute commands, and query databases.
It can make decisions: Given a goal, an Agent can decompose the task, select tools, and determine when the task is complete.
It operates through a loop: An Agent is not a one-time “input → output” process. It operates through a cycle of perception → decision → action → observation.
It maintains state: During task execution, the Agent maintains state and knows what has been completed and what happened previously.

A simple definition is:

An Agent is a software system that can autonomously use tools and make decisions through a loop to achieve a goal, with the LLM serving as the decision-making core.

There is an important point in this definition: the LLM is the decision-making core, not the entire system. This distinction is a foundation of the book and will recur throughout the chapters that follow.

1.2.4 What Is an Autonomous Agent?

An Autonomous Agent is an enhanced form of Agent that emphasizes less supervision, longer-running tasks, and more independent decision chains.

Typical characteristics include:

Long-running tasks: It can operate continuously for minutes, hours, or even days rather than handling only a few seconds of interaction.
Self-correction: When execution fails, it can detect the problem, analyze the cause, retry, or change strategy.
Goal-oriented execution: Given a high-level goal such as “build a web page,” it can decompose the goal into dozens of steps and execute them sequentially.
Weak supervision: Human involvement moves from approving every step toward approving key checkpoints or auditing the outcome afterward.

AutoGPT and BabyAGI were representative examples of the autonomous Agent wave in 2023. They attempted to let LLMs autonomously perform “planning → execution → reflection → re-planning,” but quickly exposed problems such as infinite loops, context explosion, task drift, and uncontrolled costs.

These lessons are extremely valuable for understanding Agent architecture. They were not simply failed products; they demonstrated that LLM + loop is far from sufficient and that a complete surrounding infrastructure is required. This is precisely the “Harness gap” described in Section 1.1.3.

1.2.5 What Is a Coding Agent?

A Coding Agent is an Agent specialized for software engineering and is the primary research subject of this book. Its goal is not to “chat,” but to understand code, modify code, run code, and validate results.

A Coding Agent should at least be able to:

Read code: Browse repository structures, search symbols, and locate function definitions.
Modify code: Precisely edit files rather than regenerating entire files.
Run code: Execute tests, compilation, and build commands and inspect the results.
Self-repair: Analyze failures, modify the code, and rerun validation until the issue is resolved.

Claude Code, Cursor, Codex, Cline, and Aider are examples of Coding Agents. They are valuable reference implementations for the broader Agent field because software engineering provides exceptionally rich feedback signals: compiler errors, failed tests, and type-checking results. These objective and immediate signals allow the self-correction loop to operate effectively, whereas general-purpose Agents can more easily drift or loop without meaningful progress.

1.3 Agent vs. Workflow and Traditional Automation

After defining the capability spectrum, another boundary must be clarified: how exactly does an Agent differ from a Workflow or traditional automation? This is an area where many products that are marketed as “Agents” deliberately blur the distinction.

1.3.1 Agent vs. Workflow

A Workflow is a predefined, deterministic sequence of steps:

Step 1 → Step 2 → Step 3 → Step 4

The execution path of a Workflow is determined before runtime. Even if some steps invoke an LLM, control of the overall process remains in deterministic external code. Anthropic’s Building Effective Agents describes patterns such as “Prompt chaining” and “Routing” and distinguishes them from true Agentic behavior.

The key difference is that, for an Agent, the execution path is determined dynamically by the model at runtime:

Given a goal → Model decides what to do next → Execute → Observe → Decide again

The model is not executing a predefined process; it is dynamically selecting a path.

Dimension	Workflow	Agent
Path	Predefined	Dynamically determined at runtime
Decision maker	External code	LLM
Flexibility	Low; changing requirements often requires code changes	Higher; behavior can often be changed through instructions/context
Controllability	High	Lower
Cost	Lower and more predictable	Higher and less predictable

A practical engineering rule is:

If a problem can be solved with a Workflow, do not use an Agent.

Workflows are cheaper, more controllable, and easier to test. Agents become valuable when a task genuinely requires the model to make decisions at runtime—for example, “help me diagnose this bug when I don’t know where the problem is.”

1.3.2 Agent vs. Traditional Automation

Traditional automation—RPA, scripts, and CI/CD pipelines—differs from Agents for essentially the same reason as Workflows, but with one deeper dimension: the ability to handle uncertainty.

Traditional automation assumes that the world is predictable:

Input format is fixed → the script can process it.
Page structure is fixed → the scraper can parse it.
Steps are fixed → the pipeline can execute them.

Once the environment deviates from expectations—input formats change, pages are redesigned, or unexpected exceptions occur—traditional automation often fails and requires human intervention to repair the script.

The unique value of an Agent is precisely its ability to handle uncertainty:

Input format varies → the Agent determines how to interpret it.
Environment changes → the Agent explores the new situation.
Goal is ambiguous → the Agent can clarify and decompose it.

In one sentence:

Traditional automation excels at “known repetition”; Agents excel at “unknown exploration.”

They are not substitutes. They are complementary. A mature enterprise technology stack should combine deterministic Workflows for predictable processes with Agents for uncertain tasks.

1.4 Why an LLM Can Serve as the “Brain” of an Agent

We have repeatedly described the LLM as the “decision-making core.” To understand Agents, however, we must understand what an LLM can actually do.

Many people still think of LLMs primarily as systems that “write articles” or “chat.” This significantly underestimates their potential as Agent decision engines.

This section starts from the fundamentals of Transformer architecture and gradually explains four capabilities that make LLMs suitable as the brain of an Agent:

Reasoning, Structured Output, Tool Calling, and Context Processing.

1.4.1 Transformer Fundamentals

In 2017, Vaswani et al. introduced the Transformer architecture in Attention Is All You Need, fundamentally changing the field of NLP. Its core contribution was replacing the sequential recurrence of RNNs with Self-Attention, enabling:

Parallelized training: RNNs must process tokens sequentially, whereas Transformers can process a sequence in parallel during training.
Long-range dependency modeling: Attention allows any two tokens to establish a direct relationship without the same degree of distance-related degradation found in sequential architectures.

The classic Transformer architecture is Encoder-Decoder, while modern LLMs such as the GPT family primarily use the Decoder component, forming autoregressive language models:

Input:  [t1, t2, t3, ..., tn]

Output: Probability distribution for predicting tn+1

“Autoregressive” means that the model predicts the next token based on all preceding tokens, appends the generated token to the sequence, and then predicts the next one. This seemingly simple mechanism provides the physical basis for streaming generation and step-by-step reasoning.

1.4.2 Tokens

A Token is the basic unit of text processed by an LLM. It is neither necessarily a character nor a complete word, but generally a subword-level unit.

In English, one token is roughly 0.75 words on average, while Chinese tokenization varies considerably depending on the tokenizer and model.

Tokens have two critical engineering implications that recur throughout this book:

Tokens are a billing unit: APIs commonly charge separately for input and output tokens.
Tokens measure context capacity: A model’s Context Window is measured in tokens. A so-called “context overflow” essentially means that the available token budget has been exhausted.

Understanding tokens is the first step toward understanding Context Engineering. When building LLM applications, an experienced backend engineer should always have an approximate accounting in mind:

How many tokens are currently being used? How many remain? How many tokens will each additional tool call consume?

1.4.3 Context Window

The Context Window is the maximum number of tokens a model can process as its current context. Early GPT-3 models had context windows of 4K tokens; later models expanded to 8K, 32K, 128K, and beyond, with some modern systems supporting windows reaching millions of tokens.

However, one important principle must be understood:

A larger Context Window does not eliminate the need for context management.

There are at least three reasons:

Cost: Even if a model supports a 200K-token context, sending 200K tokens on every request can significantly increase cost and latency.
Attention dilution: Research has shown that models may pay less attention to information located in the middle of very long contexts—the so-called “Lost in the Middle” phenomenon. Adding irrelevant information can therefore reduce response quality.
Signal-to-noise ratio: Feeding dozens of irrelevant files into a model can make it harder, rather than easier, to identify the information that matters.

Therefore, the size of the Context Window is only an upper bound. The real engineering challenge is deciding what should go into the window. That is the problem Context Engineering addresses in Chapter 6.

1.4.4 Attention

Attention is the core mechanism of the Transformer. Its intuition is straightforward: when processing a token, the model should focus more heavily on the parts of the context that are most relevant to that token.

Self-Attention can be summarized through three matrix operations:

Q = X · Wq

K = X · Wk

V = X · Wv

Attention(Q, K, V) = softmax(Q · Kᵀ / √d_k) · V

The Query of each token is compared with the Keys of all tokens to calculate attention weights, and those weights are then used to produce a weighted combination of Values. The result is that each token’s new representation is a weighted integration of relevant information from the surrounding tokens.

For Agent architecture, the importance of Attention is less about the mathematical details and more about one fundamental insight:

Every model output is a weighted interpretation of the current context.

This leads to two important conclusions:

Context is the model’s entire world: The model cannot see anything outside its context. Tool descriptions, file contents, conversation history, and other injected information collectively form the evidence available to it.
Context quality directly affects decision quality: Irrelevant information dilutes attention; missing critical information forces the model to guess.

1.4.5 Reasoning

Reasoning is the core capability that enables an LLM to serve as the brain of an Agent. It refers to the ability to derive unknown conclusions from known information: given a goal, the model can decompose steps, analyze constraints, evaluate alternatives, and reach a conclusion.

Modern LLM reasoning capabilities are influenced by two major mechanisms:

Emergent capabilities from scale: As model parameters and training data increase, more sophisticated reasoning capabilities can emerge.
Chain-of-Thought (CoT): When prompted to reason step by step, models can generate intermediate reasoning steps and significantly improve performance on complex tasks. This was an important finding reported by Wei et al. in 2022.

The significance of CoT for Agents is profound: it demonstrated that LLM reasoning can be externally elicited and guided. If a simple instruction such as “think step by step” can improve performance, then a carefully designed Agent loop—”observe → reason → act → observe again”—can be viewed as a structured operationalization of this idea.

Later reasoning-oriented models took this further by allocating additional internal computation before producing a final answer. This can improve performance on complex Agent tasks, but it also introduces additional cost and latency—topics discussed in detail in Chapter 28, “Long-Running Agents.”

1.4.6 Structured Output

Structured Output means requiring an LLM to produce output in a defined format, such as JSON, rather than unrestricted natural language.

Why is this important for Agents? Because an Agent is a software system, and software systems need to reliably parse model output to drive subsequent actions.

If the model produces free-form text such as “I think I should call the file-reading tool,” the application has no reliable way to execute that instruction. If instead it produces:

{
  "tool": "read_file",
  "arguments": {
    "path": "/src/main.ts"
  }
}

the program can parse the structure and execute the corresponding action.

There are three common implementation approaches:

Prompt constraints: Instruct the model to “output JSON only.” This is simple but not fully reliable because models may occasionally violate the instruction.
JSON Mode: The API provides output constraints that ensure syntactically valid JSON. This is more reliable than prompt-only constraints, but does not necessarily guarantee that the JSON conforms to the desired schema.
Constrained Decoding: During decoding, the system directly restricts the token space so that generated output conforms to a specified JSON Schema. This is the most reliable approach, but it is more complex and may introduce performance overhead.

A production-grade Agent Tool Calling system almost inevitably depends on structured output because deciding which tool to call and what parameters to provide is fundamentally a structured-output problem.

1.4.7 Tool Calling

Tool Calling is the critical leap from an LLM that can “talk” to one that can “take action.” It is also a central pillar of Agent architecture.

The mechanism works as follows: the model is trained to emit a structured tool-call request when it determines that an external action is required. The request contains the tool name and arguments. The API returns this request to the calling application—the Harness. The Harness executes the actual tool, sends the result back to the model as a new message, and the model continues reasoning based on the observation.

User: Read the version number from package.json

        ↓

Model outputs tool_call:
{name: "read_file", args: {path: "package.json"}}

        ↓

Harness executes read_file and receives the result

        ↓

Result is sent back to the model:
"The file contents are: {...}"

        ↓

Model produces the final answer:
"The version is 2.1.0"

One critical distinction must be emphasized:

The model does not actually “read the file.” It requests that the file be read. The Harness performs the real execution.

This may seem like a semantic distinction, but it is fundamental to understanding Agent security architecture (Chapter 8) and why the model itself is not the Agent (Section 1.7).

The detailed mechanics of Tool Calling, schema design, and error handling will be covered in Chapters 2 and 7. For now, the key concept is:

Tool Calling is the bridge between the language model and the real world.

1.5 The Most Basic Structure of an Agent

We can now combine the concepts introduced above and examine the minimal structure of an Agent.

User

↓

LLM

↓

Decision

↓

Tool

↓

Observation

↓

LLM

↓

Decision

...

This diagram is the minimum common denominator of Agent architecture. No matter how sophisticated an Agent framework becomes, it is fundamentally an extension and reinforcement of this loop.

The structure contains six elements:

User: Provides a goal or task.
LLM: Receives the current context and makes reasoning decisions.
Decision: The model’s output. It has two possible forms: a final answer or a request to call a tool.
Tool: Executes the action requested by the model and interacts with the real world.
Observation: The result of tool execution, returned to the model as new information.
Loop: The model makes another decision based on the new observation until it produces a final answer.

Consider a simple example: “Check today’s weather and remind me to bring an umbrella if it is raining.”

User: Check the weather and decide whether I need an umbrella

↓

LLM: (Decision) I need to call the weather_query tool with city=Beijing

↓

Tool: Returns {weather: "Moderate rain", temp: 18}

↓

Observation: Moderate rain, 18°C

↓

LLM: (Decision) It will rain today. You should bring an umbrella.

↓

Final answer

It is that simple. But notice that three fundamental Agent architecture questions are already hidden inside this example:

How does the model know which tools are available?
→ Tool Discovery is required to inject tool descriptions into the context.
Who executes the tool, how is it executed, and what happens when it fails?
→ A Tool Runtime is required.
How do we prevent the model from repeatedly calling tools without stopping?
→ Stop Conditions and Loop Control are required.

These three questions are central to the Agent Harness and will be explored throughout Part II.

1.6 The Essence of an Agent: Agent = LLM + Harness

Based on the analysis above, we can now define the essence of an Agent.

1.6.1 Two Ways to Express the Formula

The book’s central formula can be stated in one sentence:

Agent = LLM + Harness

This formula tells us that an Agent fundamentally consists of two parts: an LLM responsible for thinking, and a Harness responsible for doing—and doing so safely.

The LLM is the decision-making core. The Harness is the surrounding software that turns the model into a system capable of taking meaningful action.

There is another way to expand the formula:

Harness = Context + Tools + Loop + State + Guardrails

Substituting this into the previous formula gives the complete form:

Agent = LLM + (Context + Tools + Loop + State + Guardrails)

Why use both expressions? Because they answer questions at different levels:

“Agent = LLM + Harness” is the architectural view. It divides an Agent into the model and everything outside the model, directly supporting the argument in Section 1.7 that the model itself is not an Agent.
“Harness = Context + Tools + Loop + State + Guardrails” is the implementation view. It identifies the five core subsystems that a minimal Harness needs to provide.

Note: Chapter 4 will show that a production-grade Harness contains more than these five subsystems, including Session, Hooks, Subagents, and Observability. For now, this minimal set is sufficient to establish the core mental model.

1.6.2 What Is a Harness?

Before expanding the five subsystems, we need a clear definition of the Harness.

The original English meaning of “Harness” refers to equipment used to control and guide an animal, especially a horse. The metaphor is highly appropriate: an LLM is like a powerful but difficult-to-control horse; the Harness is the equipment used to guide and control it.

In the context of AI Agents:

A Harness is the software system surrounding an LLM that transforms it from something that can “only talk” into something that can “take action.” It provides context to the model—what the model can see; executes the model’s tool requests—what the model can do; drives the model’s decision loop—how the model continues progressing; establishes boundaries for its actions—what the model cannot do; and records and persists its behavior—what the model has done.

In one sentence:

The model is responsible for “thinking”; the Harness is responsible for “doing” and for doing it safely and reliably.

The complete discussion of the Harness—including the boundaries between Model, Agent, and Harness; the nine major responsibilities of a Harness; and why the Harness matters more than Prompt design—is reserved for Chapter 4.

For now, we will examine its five core subsystems.

1.6.3 Context

Context is the model’s working memory. All model decisions are based on the context currently available to it.

Therefore, Context Engineering is one of the most important levers for Agent capability. The same model can perform dramatically differently depending on how effectively its context is managed.

Context management is not simply about “putting more information into the window.” It involves budget management, prioritization, compression, contamination prevention, and other engineering techniques. These topics are covered in Chapters 6, 14, and 21.

1.6.4 Tools

Tools are the Agent’s “hands and feet.” The practical capability boundary of an Agent is largely defined by the tools available to it.

No matter how intelligent the model is, it cannot read or write files without file tools; it cannot access the Internet without network tools.

The core concerns of a tool system include:

How tools are defined (Schema)
How tools are discovered (Discovery)
How tools are scheduled (Dispatcher)
How tools are executed (Executor)
How results and failures are handled (Result & Error)

These topics are covered in Chapters 7, 13, and 19.

1.6.5 Loop

The Loop is the heartbeat of an Agent. It connects successive model decisions into a continuous process that moves the task forward.

The core questions are:

When should the loop continue?
When should it stop?
What should happen when execution fails?
What should happen when the context becomes full?

An interesting fact is that the Agent loop itself may contain only a few dozen lines of code, while the protective and enhancement infrastructure surrounding that loop can grow to hundreds of thousands of lines.

This is one of the most important insights of the entire book and is explored further in Chapters 3 and 20.

1.6.6 State

State transforms an Agent from a one-time, memoryless interaction into a persistent working system.

State may include:

Conversation history
Task progress
Checkpoints
Short-term and long-term memory

State persistence is fundamental to long-running Agents (Chapter 28) and production-grade Agents (Part VI).

Without state, an interrupted Agent loses its progress. With state, the Agent can resume from a checkpoint and continue running across long periods, including overnight execution.

1.6.7 Guardrails

Guardrails define the Agent’s security boundaries. They answer three questions:

What can this Agent do? What can it not do? Under what circumstances must a human intervene?

Guardrails include:

Permission controls
Sandbox isolation
Human-in-the-loop mechanisms
Audit logging

Guardrails are not optional enhancements. They are a production-readiness boundary for Agents.

These topics are covered in Chapters 8, 22, and 31.

1.7 Why the “Model Itself” Is Not an Agent

This is one of the most important sections in the chapter because it corrects a widespread misconception:

“LLMs are already very intelligent, so an LLM is an Agent.”

This misconception is harmful because it causes people to overlook the most valuable and technically complex part of Agent architecture: the surrounding software system—the Harness.

Let’s examine the issue step by step.

1.7.1 The Model Does Not Actually Access Files

When you tell an LLM, “Read this file,” it does not actually open or read the file.

It can only generate text such as, “Sure, I will read the file.”

The actual file access is performed by surrounding application code—for example, a call to fs.readFile.

The model’s knowledge of the file contents comes entirely from the surrounding system placing the file contents into the model’s context.

If the surrounding system does not provide the contents, the model knows nothing about them.

This leads to a fundamental principle:

Everything the model perceives about the real world is supplied to it by the surrounding system.

The security implication is significant. Since the model can interact with the real world only through the surrounding system, security boundaries should be enforced by that surrounding system rather than relying on the model to “behave itself.”

1.7.2 The Model Does Not Actually Execute Shell Commands

Similarly, an LLM does not execute shell commands.

If it outputs:

rm -rf /

it has only generated a sequence of characters.

The real danger begins when the surrounding system treats that output as an instruction and executes it.

This is why an Agent needs a Permission system. The model can propose a command, but whether the command is actually executed must be decided by the surrounding system.

The model is therefore the proposer, not the executor.

1.7.3 The Model Does Not Actually Modify a Database

An LLM cannot directly connect to a database, execute SQL, and modify data.

It can generate SQL text. The actual database connection and execution are performed by an external database tool.

This leads to an important architectural principle:

The model generates intent; the system executes intent.

The gap between those two is the fundamental reason the Agent Harness exists.

1.7.4 The Model Does Not Actually Operate a Browser

Browser automation is a core capability of Computer Use-style Agents, but the model itself cannot literally click a page or fill out a form.

Those operations are performed by browser automation tools such as Playwright. The model generates the intent—for example, “click this button”—while the external tool performs the actual operation.

1.7.5 The Model Does Not Automatically Bear Permission or Accountability

This is one of the easiest points to overlook and one of the most dangerous.

The model itself has no “sense of responsibility” and is not a legal entity. If an Agent deletes a production database, leaks a secret, or sends an incorrect email, responsibility lies with the people or organization deploying the Agent, not the model.

Therefore, a responsible Agent architecture must place permissions and accountability under explicit external controls:

Default deny
Least privilege
Human approval
End-to-end auditing

Summary: The Harness Is What Turns a Model into an Agent

Taken together, the five points above lead to a clear conclusion:

An LLM is an extremely capable decision engine, but it is effectively enclosed in a black box whose direct interface is text in and text out. What turns the model into an Agent is the surrounding software system—the Harness. The Harness provides context, executes tool requests, establishes boundaries for actions, and records the model’s behavior.

This conclusion is the starting point for the architectural discussions throughout the book. It explains:

Why we need a Harness (Chapter 4): because the model needs to be guided and controlled.
Why Context Engineering matters so much (Chapter 6): because the model perceives the world only through context.
Why Permission is indispensable (Chapter 8): because humans and organizations bear responsibility for executed actions, not the model.
Why the Agent Loop is simple while the Harness is complex (Chapter 3): because most of the real engineering difficulty exists outside the model.

Chapter Summary

This chapter established five core ideas that will recur throughout the book:

The evolution of Agents: From Transformer (2017) to ChatGPT (2022), from the failures of AutoGPT (2023) to the maturity of Coding Agents (2024–2025), every stage added capabilities while exposing new gaps. The lessons from AutoGPT demonstrated that LLM + loop is not enough; a Harness is required.
The capability spectrum: Chatbot (can talk) → Copilot (can assist) → Agent (can act) → Autonomous Agent (can operate more independently) → Coding Agent (can engineer software). Each stage adds another dimension of capability.
Agent vs. Workflow: The key question is who determines the execution path. If the path is dynamically determined by the model at runtime, it is Agentic behavior; if the path is predefined in code, it is a Workflow. Use a Workflow whenever a Workflow is sufficient.
Why an LLM can serve as the brain: Transformer + Attention + autoregressive generation provide the foundation for reasoning, structured output, and tool calling. Tool Calling is the bridge between the model and the real world.
The essence of an Agent: Agent = LLM + Harness, where Harness = Context + Tools + Loop + State + Guardrails. The model itself is not the Agent; the surrounding Harness is what turns the model into an actionable software system.

With these concepts in place, the next chapter will dive into the basic architecture of an LLM application. We will examine the components required to build an LLM application that can actually run, and explore how Tool Calling works at the technical level.

Please follow and like us: