Harness Engineering III: Seven Ways to Hold a Model

The Only Problem

Every coding agent harness solves exactly one problem. An LLM produces text. Your codebase is files on disk. How do you make the LLM's text reliably change the right files?

That is all of it. Everything else, repo maps, compaction, tool registries, permission systems, hooks, compound tools, prefix caching, is an answer to one of three sub problems that fall out of that single sentence.

Sub problem 1 is context. The LLM only knows what you put in the prompt. How do you give it enough information to do useful work without blowing the context window?

Sub problem 2 is edit reliability. The LLM's output is text, not code changes. How do you parse that text and apply it to files reliably, even when the LLM gets it slightly wrong?

Sub problem 3 is recovery. The LLM will fail. Files will not match. Edits will not compile. How do you detect failure and recover without human intervention?

Every architectural decision in every harness maps to one of these three problems. When you read unfamiliar code and wonder “why does this exist?”, ask which sub problem it is solving. The answer is always one of the three.

Before we go further, the question everyone asks: do any of these use LangChain, LangGraph, CrewAI, or any agent framework?

No. Not one of them. Seven codebases, zero agent frameworks.

Aider is pure Python. Its dependencies are litellm, GitPython, tree-sitter, networkx, and a handful of standard library modules. Claude Code is pure Rust (or so its architecture tells us what we can actually read is claw-code, the community clean room rewrite; more on that distinction when we get there). OpenCode uses the Vercel AI SDK, a thin provider abstraction, and Effect-TS for its effect system, but no agent framework. Copilot uses @vscode/prompt-tsx, Microsoft's own JSX-like library for assembling prompts. SmallCode, Pi, Reasonix, all built from scratch.

This is not a coincidence. The hard problem in coding agents is byte level context management. Frameworks abstract away the exact layer you need to control. Seven teams looked at the available tools and all seven reached the same conclusion: we can build this better ourselves with a simple loop. They were right.

The universal agent loop, the thing every harness implements regardless of language or complexity, is six lines:

while True:
    context   = assemble_context(session, task, files)
    response  = call_llm(context)
    actions   = parse_actions(response)
    results   = execute_actions(actions)
    session   = update_session(session, response, results)
    if done(response):
        break

The differences between harnesses are entirely in what each of these six lines does. Keep this loop in your head. It is the skeleton underneath every tool you will read about from here on.

The Scalpel (Aider ↗)

The Bet

Aider was built by one person, Paul Gauthier, starting in 2023. The philosophy is simple enough to fit in a sentence: the model should output plain text with special delimiters, not structured JSON, because generating plain text is an easier learned skill than generating valid JSON under pressure. In 2023 this was a pragmatic bet on reliability. It turned out to be prescient. Even today, Aider benchmarks better with text format edits than with tool calling for most models, and for weaker models the gap is enormous. The entire agent core is one Python class , roughly two and a half thousand lines holding everything: which files you are editing, every message from every prior turn, a compressed map of your codebase, which model to call, how much you have spent. There is no session manager. No orchestrator. It is all one object. This is not an oversight. It is the design.

The philosophy extends to the runtime. The main loop is synchronous , a literal while True with no async/await anywhere. It blocks waiting for your input, does one turn, blocks again. The turn itself is a pipeline: get input, assemble the prompt, send to the LLM, parse the edit, apply it to the file, commit to git, check whether the model mentioned files not yet in the chat and retry up to three times if so. Aider does not run in the background. It does not spawn subagents. It does not compact context. It is a pair programmer. You are always in the loop. Every successful edit gets an automatic git commit with a message written by a small cheap model reading the diff, and Aider tracks its own commit hashes so undo reverts exactly the last agent change without touching your manual commits. The git history becomes a clean audit trail of everything the agent did.

The Repo Map

The repo map is the most sophisticated piece of Aider, and the part most people do not fully appreciate. The model needs to know what is in your codebase, but you cannot send it every file. Aider solves this by building a compressed skeleton of the entire project that fits in roughly a thousand tokens. It parses every file with tree-sitter, extracts definitions and cross references, builds a directed graph of which files use which symbols, and runs PageRank to surface the files most relevant to whatever you just asked. The model sees function and class signatures only, stripped of bodies, ranked by relevance. If you say “fix the login function,” files mentioning login float to the top. The graph is cached in SQLite keyed by file modification time, so it recomputes only when files change. On a warm run the cache hit rate is around 95%. A thousand tokens to represent a hundred thousand line codebase. It is a genuinely elegant piece of engineering.

The Edit

The edit mechanism is what Aider is known for. The model outputs SEARCH/REPLACE blocks. Aider finds the SEARCH text in the target file and replaces it. The tricky part is that the LLM sometimes gets the SEARCH text slightly wrong , an extra space, a missing blank line, a ... where it elided code it assumed you would recognize. So Aider runs a sequence of increasingly forgiving matching passes: exact match first, then flexible whitespace, then it strips a leading blank line the LLM often hallucinates, then it handles those ... elision patterns. I found a fourth pass in the code, a full fuzzy matcher that computes edit distance between the SEARCH block and every window of the file, but it is unreachable , blocked by a bare return statement that has been sitting there since it was written. The function is correct. It has just never run. This is the kind of thing you only learn by reading the source.

Context is the repo map, a thousand token skeleton built from tree-sitter and PageRank. Edit reliability is SEARCH/REPLACE blocks with increasingly forgiving fuzzy matching.Recovery is git: every successful edit auto-commits, undo reverts exactly the last agent change.

Aider is a precision scalpel. One class, one loop, one job: apply LLM generated edits to a git repository as reliably as possible. The things it refuses to do , no background execution, no subagents, no hooks, no compaction, no persistence between runs , are not gaps. They are the design. Aider does one thing and bets its engineering budget on doing it perfectly.

The Control Room (claw-code ↗)

Claude Code is the opposite of Aider in almost every way. A team at Anthropic, not one person. Rust, not Python. Structured JSON tool calls, not text blocks. An event driven async runtime, not a synchronous while loop. Enterprise safety and observability, not pair programming simplicity.

A note on what I am actually reading here. Claude Code itself is closed source. What exists in the wild is claw-code, a community clean room rewrite that reconstructs the architecture from public documentation, blog posts, and observable behavior. It is not Anthropic's actual source. The architecture it reveals is directionally instructive, but the specifics are the reconstruction, not the original. I will call it claw-code from here on.

Where Aider pours everything into a single class, claw-code is a distributed system. The central runtime coordinates separate subsystems for session storage, tool execution, permissions, hooks, compaction, and usage tracking, each behind a clean interface. The turn loop is event driven: the API streams back typed events, and every tool call flows through a pipeline of hook firings, permission checks, and typed dispatch before the result is appended to session state and sent back to the model. Multiple tool calls can fire in a single turn, each going through the full pipeline.

The philosophy is visibility and control. Every tool call is a typed JSON struct, not a text block matched with string operations. Sessions persist to disk as typed message variants. The permission system has five tiers and every tool declares its required tier before it can run. Hooks fire around every tool execution. The system prompt builder walks the directory tree looking for project context files at every level, inheriting from parent directories, and injects git status and recent commits into every prompt. Subagents spawn as full independent runtimes. Everything is designed so that every component can be observed, intercepted, modified, and composed.

Context is walking the directory tree for project files and injecting git status. Edit reliability is typed JSON tool calls parsed by serde, no fuzzy string matching required.Recovery is session persistence to disk plus subagents that spawn as full independent runtimes.

claw-code is a control room. It is built for enterprise teams running autonomous agents over long periods, where you need to see what is happening, control what is allowed, and persist everything to disk. The philosophy is the opposite of Pi's: trust nothing, verify everything, and build the infrastructure to do it at scale.

The Lighthouse (OpenCode ↗)

OpenCode was built by the SST team. The philosophy: the agent is a local HTTP server and the UI is just a client. Any client. The TUI, a desktop app, a VS Code extension , anything that speaks HTTP can drive the agent. The study documents describe a Go TUI with Bubble Tea and a TypeScript/Bun server using Hono. There is no Go code anywhere in the repository. The entire project is TypeScript running on Bun. The TUI is Solid.js, not Bubble Tea. The HTTP layer is Effect-TS, not Hono. The architecture the documents describe is conceptually accurate. The implementation is entirely different.

The killer feature is LSP integration. OpenCode auto detects your project's languages by scanning for markers like go.mod or Cargo.toml, starts the appropriate language server in the background, and injects live type errors, function signatures, and references into every prompt before the model writes a single line. I counted around 40 language server definitions. The agent system is elegantly simple: agents are configuration objects, not classes. Build gets full permissions, Plan gets read only , adding a new agent type means adding a config. Sessions persist to SQLite via Drizzle ORM. Restarting OpenCode does not lose your conversation.

Context is LSP diagnostics injected into every prompt before the model writes a single line, plus AGENTS.md project context. Edit reliability is standard typed JSON tool calls through the Vercel AI SDK. Recovery is SQLite session persistence so restarting never loses the conversation.

OpenCode is a lighthouse. It runs on your infrastructure, speaks HTTP, and any frontend can connect. LSP native from the start, model agnostic by principle, fully air gappable with Ollama.

The Native (Copilot ↗)

GitHub Copilot started as autocomplete. Everything since , chat, agent mode, shell commands , was layered on top of an extension already deeply integrated into VS Code. Copilot is three different systems sharing the same extension: inline ghost text that appears as you type (no tool calls, no loop), a chat panel in the sidebar for Q&A about your code, and a full agent mode with tool calling and multi step reasoning across the workspace.

The most architecturally interesting piece is @vscode/prompt-tsx, a JSX like library for assembling prompts. Each component renders itself to tokens, the framework fits everything within budget, and components with lower priority get dropped first. It is the most compositional prompt assembly system of any harness. Copilot has four session backends: local agent, CLI, cloud (your code goes to GitHub's servers), and Claude (routing through Anthropic's Agent SDK).

Because Copilot lives inside VS Code, it sees things no terminal tool can: already running LSP servers, your open files, your cursor position, the workspace semantic index. In exchange it gives up everything else , no terminal, no SSH, no desktop app.

Context is the VS Code workspace index, open files, cursor position, and running LSP servers. Edit reliability is standard typed tool calls. Recovery depends on which of the four session backends you are using, from local agent loop to cloud delegation.

It is a native. The deepest IDE integration available because it is part of the IDE. And it cannot leave.

Copilot is the one harness here I did not clone and read line by line. Its extension is open source but sprawling, and I am reporting its architecture from the source structure and documentation rather than the same deep verification I gave the others.

The Exoskeleton (SmallCode ↗)

The Philosophy

SmallCode is the only harness designed from the ground up for small local models in the 4B-7B parameter range. Where every other harness assumes a smart model and builds infrastructure around it, SmallCode assumes a limited model and compensates with an intelligent harness. The philosophy in one line: the harness does the heavy lifting so the model does not have to be smart.

Compound Tools

SmallCode's most important innovation is the compound tool. Instead of giving the model four separate tools , find a file, read it, edit it, verify the edit , SmallCode gives it one tool that does all four internally. The model makes one decision instead of four. This matters because small models lose coherence after three or more sequential tool calls. Each call is a new generation; the model has to remember why it is making the call, what it is looking for, and what it plans to do with the result. Collapse the chain into a single compound operation and the model stays inside its effective reasoning range. The documents got the tool names wrong, but the concept is exactly right.

The Improvement Loop

The second innovation is the improvement loop. After every code write, SmallCode automatically runs verification and feeds errors back to the model. A 4B model that cannot produce correct code cold can fix a TypeError when shown the traceback. The model's job becomes “produce code that passes the linter,” not “produce correct code in one shot.” It is dramatically easier. The study documents describe automatic revert on each failure; the actual revert only fires on escalation failure, a subtle but meaningful difference in how recovery works.

Decomposition and Escalation

When a task fails twice, SmallCode does not retry the same approach. It decomposes: splits the file, rewrites a section, extracts a function, tackles one error at a time. The documents claim this produces independent subtasks with lineage tracking. The actual code produces strategy changes for the same file , the parent ID in the codebase is for distributed tracing, not task tracking. But the idea is sound: reduce scope and the model often succeeds immediately.

Several claims in the study documents turned out to be wrong. The code graph is a regex based symbol extractor, not a tree-sitter graph with call relationship tracking. The token budget is a dynamic 70% of the context window, not a hard coded 2,000 tokens. But the cloud escalation system is real and well implemented: when local model and decomposition both fail, and the user has configured an API key, the task escalates to a cloud model with a configurable per session cap.

Context is a dynamic token budget at 70% of the model's context window. Edit reliability is compound tools that collapse multi step chains into single decisions, plus an improvement loop that feeds compile errors back for iterative fixing. Recovery is decompose on failure, then cloud escalation with a configurable per session cap.

SmallCode is an exoskeleton. The harness wraps around the model and provides capabilities the model itself lacks. Compound tools collapse chains, the improvement loop turns one shot generation into iterative refinement, decomposition breaks impossible tasks into possible ones, and escalation catches what remains. The philosophy extends to what it does not do: the escalation cap prevents runaway cloud costs, the reverts are conservative, the token budgets are dynamic. Every feature is a harness level compensation for a known small model failure mode.

The Purist (Pi ↗)

The Philosophy

Pi was built by Mario Zechner as a deliberate reaction against complexity. The philosophy: give developers full control of context engineering; do not inject anything behind their back. Of all seven codebases, Pi had the highest fidelity between documentation and reality , every claim I checked was confirmed in the code.

The core agent loop is simple enough to hold in your head: stream from model, for each event, if it is a tool call, validate the arguments, execute, append the result to context, loop. No permission system. No hooks. No compaction. Just the loop. Sessions are stored as trees , every entry has a parent ID, you can branch at any point, try different approaches from the same starting point without losing either branch. This is better than linear history for anything where exploration matters.

The Extension Model

Extensions auto discover from a directory. Drop a file in and it loads , tools, event handlers, commands, shortcuts, CLI flags, message renderers, provider registrations. No configuration needed. Pi's most important contribution might be its “no” list, documented explicitly in the coding agent README: no MCP, no sub-agents, no permission popups, no plan mode, no built-in to-dos, no background bash. For each, it tells you the alternative - use tmux, write plans to files, build it as an extension. This list is a design document. It tells you exactly what the author prioritized over what he did not. More projects should have one.

Little-Coder, built on top of Pi, demonstrates the model in practice: 30 skill markdown files and 28 extension directories. Skills are plain text instruction documents for specific task types, injected into the system prompt only when relevant. The model gets domain expert guidance for the operation it is about to perform without that guidance bloating every turn.

Context is entirely user controlled, the harness injects nothing behind your back. Edit reliability is TypeBox validated JSON tool calls. Recovery is tree based sessions where you can branch at any point and try different approaches without losing either.

Pi is a purist. Trust the model. Keep the harness out of the way. Make everything an extension. Document what you will not build.

The Accountant (Reasonix ↗)

Cache Economics

Reasonix is engineered around a single economic fact: DeepSeek bills cached input at 10% of the miss rate. If you can achieve 99%+ cache hits, you pay ten times less for long sessions. Every architectural decision traces back to this property.

Reasonix is written in Go.

The 3 region context model is Reasonix's most transferable idea. Region one is the immutable prefix , system prompt + tool specs + few shot examples , hashed and pinned at session start. It never changes between turns, guaranteeing a cache hit on the largest part of the context. Region two is the append only log , the conversation history that grows per turn but never gets rewritten. Prior turns are preserved byte for byte, so they remain cache hits. Region three is volatile scratch , per turn state that gets discarded after each turn and never sent upstream. R1 reasoning content lives here. If it is harvested, the useful structure is extracted cheaply by a flash model and the raw noise is discarded.

This discipline , never rewrite history , is what makes the cache math work. Most agent loops reorder or rewrite context between turns (injecting timestamps, reordering messages, modifying tool descriptions). Every mutation destroys cache hits. Reasonix makes the regions immutable specifically to guarantee cache stability.

The cache math matters. A real user case from the docs: 435 million input tokens in one day. With 20% cache hit rate (typical unoptimized agent), 80% is billed at full price. With 99.82% cache hit rate (Reasonix), 0.18% is billed at full price. Cost: $12 vs. $61 for identical work. Over a month running continuously: ~$360 vs. ~$1,830.

The tool parallel safety system is well designed. Tools declare whether they are read only, the dispatcher groups consecutive reads into parallel batches capped at 8, and writers execute one at a time. Safe because reads are idempotent and writes are not. Four repair passes handle the reality that DeepSeek's streaming sometimes produces broken output: a storm breaker that redirects the model after repeated failures, a repeat guard that blocks duplicate writes from the same turn, a message normalizer that repairs truncated JSON and incomplete brackets, and a reasoning content extractor that harvests R1 reasoning for display without re-uploading it to the API.

Context is a 3 region model with an immutable prefix and append only log, guaranteeing cache hits by never rewriting history. Edit reliability is typed tool calls with four repair passes for DeepSeek's streaming quirks. Recovery is a storm breaker that redirects the model after repeated failures, plus a repeat guard that blocks duplicate writes.

Reasonix is an accountant. Every byte of context is engineered for cache stability. Cost is a first class architectural concern, not an afterthought. If your agent runs for minutes, cost does not matter. If it runs for weeks, cost is existential.

The Comparison

Eight Things I Now Know

1. No one uses agent frameworks. There is a reason.

Seven codebases. Zero LangChain. Zero LangGraph. Zero CrewAI. The hard problem is byte level context management. Frameworks abstract away the exact layer you need to control. Every team reached the same conclusion independently. Build your own loop. It is 8-20 lines. Understand every line. Then add complexity only when a specific failure mode demands it.

2. The edit format decision is the fork in the road.

There are exactly two ways an LLM can express a code change: text blocks with delimiters, or structured JSON tool calls. Every other architectural decision flows downstream from this choice. Text blocks work on any model but require fuzzy matching. JSON tool calls are type safe but require a model trained on function calling. Small models fail at JSON more often. Aider's bet on text blocks , made in 2023 when tool calling was unreliable , turns out to be prescient for the small model use case. If you are building for local models, start with SEARCH/REPLACE blocks.

3. Context management is the whole game.

Every harness partitions the context window into the same five regions: system prompt, project context, file content, conversation history, current request. The differences are entirely in how each region is filled and at what size. Aider's repo map fits the whole project into ~1,000 tokens using tree-sitter and PageRank. Reasonix's 3 region partition guarantees cache hits by making the prefix immutable and the log append only. SmallCode's explicit token budgets prevent any region from starving the others. The quality of a harness is the quality of its context allocation.

4. Verification is not optional. It is the contract.

Every serious harness has a deterministic verification step. The LLM's confidence about whether something worked is meaningless. Only a programmatic check counts. Aider commits only after successful edit apply. SmallCode compiles and lints after every write. Reasonix verifies tool results before log entry. If your harness does not verify, it is a toy.

5. The improvement loop is the highest-ROI feature for small models.

SmallCode taught us this. After every edit, run verification and feed errors back. A 4B model that cannot produce correct code cold can fix errors when shown a traceback. The model's job becomes “produce code that passes the linter,” not “produce correct code in one shot.” It is dramatically easier. Three attempts with error feedback can turn a rough first pass rate into something far higher, at essentially no additional cost on local hardware.

6. Compound tools answer context coherence degradation.

Small models lose coherence after 3+ sequential tool calls. Each call is a new generation; the model forgets why it is making the call. Collapse four calls into one compound tool and the model makes one decision instead of four. The harness handles the chain internally. SmallCode's actual tools show the pattern: read_and_patch (find a file and edit it in one step), search_and_read (locate and pull in relevant code), create_and_run (write something and immediately execute it). The shape that matters is bundling a multi step operation behind a single decision.

7. Decomposition is how you handle the genuinely hard.

When a task fails twice, do not retry the same approach. Break it into smaller pieces. A small model that fails on a complex task is not stupid , it is overwhelmed. Reduce the scope and it often succeeds immediately. Task granularity is a parameter you can tune at runtime. The harness should tune it automatically.

8. The philosophy you choose determines everything you do not build.

Pi's “no” list is the clearest example. No sub-agents. No MCP. No background bash. No plan mode. Each of these is a choice to prioritize simplicity over power. Aider's choice to be stateless is a choice to prioritize pair programming over autonomy. Reasonix's choice to make cost a first class concern is a choice to prioritize session duration over feature breadth. The things you refuse to build define your harness more than the things you build.

Every coding agent harness is a solution to one problem. An LLM produces text. Your codebase is files on disk. Seven teams built seven different answers. They are all, in their own way, correct.