The third layer. The harness is everything around the model: tools, state, permissions, memory, sandboxes, retries, observability. A model can only be as reliable as the environment it runs in. When an agent lacks a capability, that is never a prompt problem — it is a harness problem. Ten lessons — including the three-layer diagnostic (which layer owns the failure), hooks-vs-prompts, and the permission-surface audit.
A tool is a structured-output schema the model fills in: name, parameters, and a description that tells the model exactly when to use it. Overlapping tools create wrong choices; vague parameter names create garbage arguments. Write tool descriptions with the same care as prompts — they are prompts, injected on every turn.
The 12-factor rule: the framework shouldn't decide what happens next — your code should. Keep execution state (what step we're on, what's done) unified with business state (the actual data), so a crash or pause loses nothing. An agent you can launch, pause, and resume with simple APIs is an agent you can operate; one that lives inside a framework's hidden loop is one you can only restart.
When a tool fails, the worst harness swallows the error; the best compacts it into the window so the model can react — retry, work around, or ask for help. Same for limits: timeouts, retry budgets, and spend caps belong in the harness, not in hope. An agent that never sees its own failures repeats them.
Decide what the agent can do before deciding what it should do. Read-only by default; writes gated; irreversible actions (send, delete, pay) behind human approval. Sandboxes make ambition safe: an agent with a scratch directory and no production keys can try things an unrestricted agent must never attempt. Small blast radius is what lets you grant autonomy.
Every run should leave a trace: which tools fired, with what arguments, costing what, taking how long. That's how you debug the layer correctly — missing capability is harness, unreliable completion is loop, wrong order is graph. Without traces, every failure looks like “the model was dumb,” and you fix the wrong layer.
Harness, loop, and graph are not competing approaches — they nest. The graph runs inside the harness; the loops live inside the graph’s nodes; and the harness supplies the state, tools, and evaluators those loops consume. One phrase holds the whole stack: the harness is the environment, loops are the feedback, the graph is the flow. And one test finds your harness instantly: remove the model from your architecture diagram — everything left (tools, data access, state store, sandbox, evaluators, retry policy, UI) is the harness. Two teams on the same model get very different outcomes because the working conditions differ, not the intelligence.
Each layer is a different lever to pull when the system fails. Fix the layer that owns the failure — a model cannot reliably compensate for stale state, ambiguous tool schemas, broken APIs, or missing exit conditions.
| Symptom | Layer | Likely fix |
|---|---|---|
| Can’t access the right data or tool safely | Harness | Tool contract, permissions, sandbox |
| Forgets progress across sessions | Harness | Durable state, checkpoints, progress artifacts |
| First attempt close but not reliable | Loop | External grader, deterministic tests, bounded retry |
| Keeps working after success, or stops before proof | Loop | Evidence-based stop rules, budget limits |
| Specialists must run in a controlled order | Graph | Explicit nodes, edges, routing, joins |
| Failures hard to locate in a multi-step process | Graph + harness | Traces aligned to nodes and transitions |
| Workflow changes too often for a fixed diagram | Simpler harness | Keep control model-driven; delay the graph |
The most useful row is the last one: when the model must invent the plan dynamically, forcing every path into a diagram makes the system more brittle, not less. A graph can freeze assumptions too early — formalize the stable paths you observe, not the process you imagine.
The failure patterns behind most weak agent architectures: building the graph before understanding the work (start with traces from a simpler harness, then formalize the stable paths); letting the same model write and grade without safeguards (shared blind spots — prefer deterministic checks, separate reviewer context, human gates for high-impact actions); “keep trying” as a loop specification (an unbounded retry loop is a cost leak — every loop needs a measurable objective, fresh evidence, a retry cap, and a named escalation path); treating the harness as a dumping ground (more tools raise selection errors, noisy context raises confusion, broad permissions raise risk); and blaming the model for orchestration failures.
The cleanest line in harness design: a hook is a deterministic rule the runtime enforces — it fires every time, no exceptions — while a prompt instruction is a suggestion the model usually follows but can reason around. “Before every commit, run the linter” as a hook is enforcement; the same sentence in CLAUDE.md is a wish (Issue #008’s rule wearing harness clothes: asserted is not enforced). The test: if the behavior must happen every time, it’s a hook; if it needs the model’s judgment, it’s a prompt. The same discipline applies on the way out: every response carries a stop_reason, and production code must read it — end_turn means finished, max_tokens means your output was truncated (incomplete JSON cut off mid-string is its signature), tool_use means the model wants a tool. An agent that stopped is not an agent that finished, and “just use the output” is the wrong answer that sounds like engineering.
Permission models have surface area, and the breach usually comes through the tool you forgot. Rules evaluate deny → ask → allow with the most restrictive winning, and scopes stack (an org-managed deny beats a project allow) — but the real traps are equivalences: file reading is several separate tools with separate permissions, so denying two paths to the file still leaves the third open; wildcard syntax has teeth (bash(ls *) matches arguments, bash(ls*) matches command-name continuations like lsof); and a sandbox that wraps only shell commands does nothing about fetch/read/search tools that operate outside it. Then the finding that should reorder your threat model: run an agent with permission checks skipped and it can disable its own sandbox when it decides the sandbox is blocking the task — demonstrated live in Andrew Brown’s exam-prep labs: sandbox on, command denied, and the agent’s reasoning read “let me retry with sandbox disabled.” Two layers of protection, both removed by the thing they were protecting against. A guard the agent can modify is a suggestion; enforcement must live where the agent cannot reach it.