What is an Agent Harness?
An agent harness is the entire execution layer surrounding an AI model β the system prompts, tools, memory, sandboxes, middleware, and orchestration logic that turn a language model into a working agent. If you're not the model, you're the harness.
Inside the Harness
A raw model takes text in and produces text out. It cannot access files, call APIs, maintain state, or execute code. The harness provides all of these capabilities β it is the scaffolding that makes model intelligence useful.
System Prompt
Defines the agent's role, capabilities, and behavioral constraints.
The system prompt is the harness's first layer of control β it shapes how the model interprets every user request and which strategies it applies.
Tools & Skills
APIs, databases, web search, file systems, code execution β anything the agent can use.
Tools are registered with descriptions so the model knows what's available. Skills bundle related tools and instructions for progressive disclosure.
Orchestration Logic
The agent loop, subagent spawning, model routing, and task decomposition.
The harness runs the ReAct loop β reason, act, observe, repeat. It manages when to spawn subagents, which model to route to, and when to stop.
Memory & State
Conversation history, context files, and persistent knowledge across sessions.
The filesystem is the foundational memory primitive. Files like AGENTS.md let agents durably store and retrieve knowledge across sessions.
Sandboxes
Isolated execution environments for running agent-generated code safely.
Sandboxes give agents a secure workspace to run code, install dependencies, and inspect results without risking the host environment.
Middleware & Hooks
Deterministic logic that fires before/after model calls and tool executions.
Middleware handles compaction, retries, guardrails, human-in-the-loop approvals, and cost controls β anything that needs to run at a specific point in the loop.
The Agent Loop
# The harness runs the agent loop
def run_agent(model, tools, user_message):
messages = [system_prompt, user_message]
while True:
# 1. Harness provides context to the model
response = model.generate(messages, tools=tools)
if response.tool_calls:
# 2. Harness executes tool calls
for call in response.tool_calls:
result = execute_tool(call)
messages.append(result)
else:
# 3. Model is done β return answer
return response.text
The model decides what to do. The harness decides how to do it. This separation is what makes agents flexible, testable, and swappable.
β Vivek Trivedy, "The Anatomy of an Agent Harness" (LangChain, 2026)
Middleware: How You Shape the Harness
Middleware hooks into the agent loop at each step β before and after model calls, before and after tool calls, at startup and teardown. Each piece handles one concern and composes freely with any other. This is how you tailor a harness to your specific task.
Context Management
Prevent context overflow and keep the model focused on what matters.
- Summarization middleware compacts long conversations
- Context editing removes noisy or irrelevant messages
- Tool output offloading keeps large results out of context
Memory & Learning
Load knowledge at startup, persist learnings at the end of a run.
- AGENTS.md files injected into context on start
- Skills loaded progressively as needed
- Filesystem middleware for durable state
Safety & Guardrails
Enforce policies that must fire on every call regardless of model behavior.
- PII detection and redaction
- Human-in-the-loop approval gates
- Output content filters and action limits
Cost Controls
Prevent token spend from accumulating unchecked.
- Model call limits per task
- Tool call rate limiting
- Prompt caching for long-running tasks
Error Recovery
Handle transient failures with retry logic and fallbacks.
- Tool retry with exponential backoff
- Model fallback when primary is unavailable
- Stream error handlers
Observability
See exactly what the agent is doing at every step.
- Trace logging for every model call and tool invocation
- Stream handlers that route events to different consumers
- Custom state tracking across middleware hooks
from langchain import create_agent
from langchain.middleware import (
SummarizationMiddleware,
HumanInTheLoopMiddleware,
ToolRetryMiddleware,
)
agent = create_agent(
model="anthropic:claude-sonnet-4-6",
tools=tools,
middleware=[
SummarizationMiddleware(),
HumanInTheLoopMiddleware(approval_tools=["send_email"]),
ToolRetryMiddleware(max_retries=3),
],
)
Middleware bundles related logic in composable, sharable units. The same middleware can be reused across every agent in an organization β new agents inherit battle-tested behavior without rebuilding it.
β Sydney Runkle, "How to Build a Custom Agent Harness" (LangChain, 2026)
Guides vs. Sensors
A well-built harness serves two goals: it increases the probability that the agent gets it right in the first place (guides), and it provides feedback loops that self-correct issues before they reach human eyes (sensors).
Steer the agent before it acts. Guides increase the probability of good results on the first attempt.
Observe after the agent acts and help it self-correct. Especially powerful when optimized for LLM consumption.
Separately, you get either an agent that keeps repeating the same mistakes (feedback-only) or an agent that encodes rules but never finds out whether they worked (feedforward-only). You need both.
β Birgitta BΓΆckeler, "Harness Engineering" (Martin Fowler, 2026)
Why Harness Engineering?
Task-Harness Fit
A harness for a customer service agent looks very different from one built for a long-running coding agent. The best agents aren't just built with capable models β they're built with harnesses that tightly fit the task.
Model Independence
Because the harness is separate from the model, you can swap between GPT-5, Claude, Gemini, or open-source models without rewriting agent logic. The harness abstracts the model layer.
Composability
Middleware, tools, and skills compose freely. Each piece handles one concern. The same memory middleware works across every agent. New capabilities are one middleware away.
Observability & Trust
Every decision is traceable. See what the model reasoned, which tools it invoked, what data it accessed, and why. Essential for debugging, compliance, and building trust in autonomous systems.
Frequently Asked Questions
What is the difference between a model and a harness?
The model is the AI β the language model that reasons, plans, and generates text. The harness is everything else β the system prompts, tools, memory, sandboxes, orchestration logic, and middleware that wrap around the model. A model alone can only produce text. A harness turns it into an agent that can take actions in the real world.
Why not just put everything in the system prompt?
System prompts are static text. They cannot execute code, call APIs, manage state, or enforce policies dynamically. A harness provides runtime capabilities β it can execute tool calls, manage context overflow, retry failed operations, and enforce guardrails regardless of what the model tries to do. Some things cannot and should not live in a prompt.
What is middleware in an agent harness?
Middleware is a customization layer that hooks into the agent loop at specific points β before and after model calls, before and after tool executions, at startup and teardown. Each piece of middleware handles one concern (compaction, retries, guardrails, logging) and composes with others. It is how you tailor a generic harness to a specific task.
Can I use the same harness with different models?
Yes. The harness and model are separate layers. You can swap from GPT-5 to Claude to an open-source model without changing your tools, memory, middleware, or orchestration logic. This model independence is a core design principle of the harness pattern.
What are guides and sensors in harness engineering?
Guides are feedforward controls that steer the agent before it acts β system prompts, coding conventions, architecture docs, and skills. Sensors are feedback controls that observe after the agent acts and help it self-correct β linters, test suites, code review agents, and mutation testing. A good harness needs both to prevent mistakes and catch them when they happen.
What is task-harness fit?
Task-harness fit is how well your harness matches the actual demands of the task β the context it needs, the failures it will encounter, the policies it must enforce, and the environment it operates in. A coding agent needs filesystem access, sandboxes, and test runners. A customer service agent needs memory, approval workflows, and CRM integration. The best agents are built with harnesses tightly fitted to their mission.
Ready to Build with Harnesses?
Create an AI agent with a custom harness tailored to your business. Connect your tools, configure middleware, and let the agent handle the rest.