Harness Engineering IV: Whatever Exists

Whatever exists, he said. Whatever in creation exists without my knowledge exists without my consent.

Every coding agent harness solves one problem. An LLM produces text. Your codebase is files on disk. How do you make the text reliably change the right files? Everything else, repo maps, token budgets, edit formats, verification, is an answer to one of three sub problems: what to put in the prompt, how to apply the change, and what to do when it fails. Context, edit reliability, recovery.

This past week I built the first of those three for Grit. The vision layer. The part of the harness that sees the codebase, finds the relevant code, and hands it to the model. The model is kept small and dumb on purpose, a 7 or 8 billion parameter quantized model running locally, while the harness carries all the intelligence. The harness finds the code. The model rewrites one function at a time. This is the story of how the harness sees, and what broke when I asked the model to see with it.

The judge's principle runs underneath the whole design. Nothing in the codebase can be acted on until it has been seen and known. What the harness has not indexed does not exist to it, and what does not exist cannot be changed. So before the model touches a single line, the harness surveys the entire territory. Every function, every class, every call site, mapped and recorded. Nothing acts without the harness's knowledge.

The Toolbox

Before how Grit sees, a word on what it sees with. Most of the tools are borrowed. I studied seven production harnesses, Aider, Claude Code, OpenCode, Copilot, SmallCode, Pi, and Reasonix, and took what worked.

tree-sitter is a parser. Give it source code in any language and it builds a concrete syntax tree: every function, every class, every call site, down to the byte position in the file. Unlike anything regex based, it does not break on a syntax error. A missing bracket does not stop it from parsing the rest of the file. Aider uses tree-sitter to build a repo map, a thousand token skeleton of the whole codebase. Grit borrows the idea and extends it. We store byte ranges on every node, not just line numbers, because byte precise positions are what make whole function replacement possible later.

ripgrep is search, like grep but an order of magnitude faster on a large codebase. Grit shells out to it for full text search. It is faster than anything I could write in Python and it is one static binary with no dependencies. Search for "login redirect" across a hundred thousand lines and it returns in milliseconds. No persistent index required at this scale.

SQLite is the database. The symbol graph, the edge graph, and every file's modification time live in a single .db file inside the project's .grit directory. Each entry is keyed by file path and modification time. On a warm run only the files that changed get re parsed. Aider's repo map hits a 95% cache rate this way. Grit targets the same.

Three tools. All deterministic, all local, none of them needing a network call or an API key. And here is the first rule of Grit. The model never touches any of them.

The model never explores. It never runs a search, never reads a file it chose, never follows a trail of its own. The harness does all of it and hands the model the one function it needs to edit. This is the opposite of how most agents work. Claude Code walks the directory tree injecting project context. Copilot leans on the editor's workspace index and a running language server. OpenCode starts language servers for forty languages. Grit rejects model driven exploration outright. The harness finds. The model edits.

Three more choices worth naming, because every major harness makes the opposite call.

No LSP as a primary dependency. OpenCode and Copilot both wire language servers in deep, injecting live type errors and signatures into every prompt. But a language server needs the toolchain installed and the project in a compilable state, and those conditions fail constantly on the throwaway machines and half refactored codebases where Grit is meant to run. Grit uses LSP when it happens to be there and works completely without it. tree-sitter and ripgrep always work.

No agent frameworks. No LangChain, no LangGraph, no CrewAI. None of them appear in any of the seven harnesses I studied. The hard problem in a coding agent is byte level context management, and a framework abstracts away the exact layer you need to hold in your hand. Every team reached the same conclusion on its own. Build your own loop. It is eight lines. Understand every one of them.

How the Harness Reads Code

When Grit first enters a repo it builds two structures in SQLite.

The symbol graph. tree-sitter parses every file into an AST. From that tree Grit pulls every function and class definition, every call site, every import. Each definition stores not just its line numbers but its exact byte range, the offset from the first byte of def to the byte after the last line of the body. Those byte ranges are the key to everything downstream.

The edge graph. A reference in file A to a symbol defined in file B makes a directed edge A→B, weighted by how interesting the identifier is. The rules are tuned for Python for now. A snake_case name longer than eight characters gets a 10× boost, because it is distinctive, resolve_redirects means more than run. A name starting with an underscore gets a 0.1× penalty, private by convention. A name defined in more than five files gets a 0.1× penalty, because it is a common word. What is left is a graph of who calls who and who imports what. The architecture is language agnostic, tree-sitter has grammars for over a hundred languages, but only the Python grammar is wired up today. The weighting would need different rules for camelCase TypeScript or kebab-case Clojure.

Both structures are cached by file modification time. When Grit re indexes later, after a few edits, it checks each file's current mod time against the stored one. For the 95% of files that did not change it reuses the cached parse. The 5% you actually edited get re parsed. On a thousand file project where you changed three functions, indexing takes milliseconds instead of minutes.

Then a task arrives. Say: users report that calling get_user_name with no user causes a crash. The search layer fires seven signals at once.

Here is how each signal works:

1. Keyword → symbol (weight 0.8)

You type "redirect copies original request." Grit splits that into tokens: redirect, copies, original, request. Then it queries the definitions table: give me every function and class whose name contains any of these words. resolve_redirects matches on redirect. SessionRedirectMixin matches on redirect. prepare_request matches on request. Each match gets a score proportional to how many tokens hit.

2. Keyword → path (weight 0.4)

Same tokens, matched against file paths instead of symbol names. "redirect" hits src/requests/sessions.py because, well, it does not, actually, there is no "redirect" in that path. That is why this signal is weighted low (0.4). It catches cases where the file is literally named after the feature: login.py for a login bug, validation.py for a validation bug. When it fires, it is a strong signal. It just does not fire often.

3. Keyword → content (weight 0.6)

This is ripgrep. Grit shells out to rg with the most distinctive query terms and gets back every line in the codebase containing those words, with file and line number. Unlike the symbol search (which only checks names), this finds the words anywhere, in comments, in string literals, in the middle of a 400 line function body. Higher weight than path matching because a content hit means the concept actually appears in the file, not just in its name.

4. Lexical overlap (weight 0.9), the BM25 one

This is the signal that fixes disambiguation. Here is the problem it solves:

You type "age validation accepts zero." The keyword→symbol signal finds is_valid_age (in validation.py) and is_adult (in users.py). Both match on "age." Which one is right?

Lexical overlap tokenizes everything, the query, file paths, and symbol names, by splitting on snake_case, camelCase, dots, and slashes. Then it computes how many tokens overlap between the query and each candidate's (path + symbol):

Query tokens: {age, validation, accepts, zero}

Candidate A:
path: src/validation.py → tokens: {src, validation, py}
symbol: is_valid_age → tokens: {is, valid, age}
Path overlap with query: {validation} = 1 match / 4 query tokens = 0.25
Symbol overlap: {age} = 1 match / 4 query tokens = 0.25
Score: 0.7 × 0.25 + 0.3 × 0.25 = 0.25

Candidate B:
path: src/users.py → tokens: {src, users, py}
symbol: is_adult → tokens: {is, adult}
Path overlap: {} = 0
Symbol overlap: {} = 0 ("adult" ≠ "age")
Score: 0

Candidate A wins because validation is in the file path and age is in the symbol name. Candidate B has neither, "adult" does not match "age" (different words) and "users" does not match "validation." The 70/30 split means a path match (the file is literally called validation.py) counts more than a symbol match (the function happens to contain "age").

This is BM25 at heart, a ranking function from information retrieval that scores documents by how many query terms they contain, weighted by how rare those terms are. "Validation" is a rare, distinctive word; matching a file path is a strong relevance signal. We do not need the full BM25 machinery (inverse document frequency over the entire codebase) because at this scale, simple Jaccard similarity with path and symbol weighting is enough.

5. Structural spread (weight 0.5)

Once a file gets a hit from signals 1 through 4, Grit follows the edge graph one or two hops outward. If sessions.py scored high, and the edge graph says sessions.py imports from models.py and auth.py calls into sessions.py, those neighboring files get a small score bump. They do not match any keywords, but they are structurally related to code that does. This catches the router config that a login function depends on but that never mentions "login" in its own name.

6. Recency (weight 0.3)

Files touched in the last 10 git commits get a flat 0.5 score. Low weight because recency is a weak signal, the fact that you edited something recently does not mean it is related to the current bug, but it is cheap to compute and sometimes surfaces the right file when all other signals are silent.

7. Error traceback (weight 1.0)

If a previous edit attempt failed, Grit parses the Python traceback or pytest output to extract (file, line_number) pairs. The innermost frame (where the error actually occurred) gets score 1.0. Outer frames get diminishing scores. Weighted at 1.0 because a traceback is almost never wrong, if the stack says the error is at sessions.py:312, that is the file you need to look at.

Each signal produces a candidate to score mapping. Scores normalize to 0-1, combine with the weights above, and sort with tiebreakers that favor a specific symbol match over a file level one. A definition boost of 2.0 makes sure the file where a function is defined outranks files that merely reference it. The top candidate feeds the next stage. The model was never asked.

There is a gap here. All seven signals are lexical. The user's words have to appear somewhere in the code. "Make it faster" will not match optimize_query no matter how good the token splitting gets. The eventual fix, once the current approach hits a measured failure rate, is a semantic layer: embed every symbol definition at index time with a small local model, then match the task against the vector space of names and signatures. Something like nomic-embed-text runs locally at around 400 embeddings a second, fast enough to index a Flask sized repo in seconds. But the question is whether vocabulary mismatch is actually the binding constraint, and so far lexical search finds the right file in the top three on every test case I have. When it stops doing that, and the failure is clearly a naming gap and not a structural one, that is the trigger. Not before.

There is a second future signal. The disambiguator: when all seven signals fire and two candidates have near identical scores, the harness asks the model a multiple choice question. Not "find the bug in this codebase." Not "read these files and tell me what is wrong." Just:

Query: "age validation accepts zero and negative numbers"

Candidate A: src/validation.py :: is_valid_age
    def is_valid_age(age):
        return isinstance(age, int)

Candidate B: src/users.py :: is_adult
    def is_adult(user):
        age = user.get("age", 0)
        return age > 18

Which function should be changed? Answer A or B.

The model sees three things: the original search query, the symbol names, and a 3 line code snippet for each. No file contents. No repo context. No ability to ask for more. It outputs "A" and the harness proceeds. The model's entire role in the vision layer is breaking a tie the harness could not resolve deterministically. It never explores. It never reads a file. It picks from a shortlist.

This is not built yet because lexical matching has not produced a tie we could not break with the definition boost and tiebreakers. When it does, this is what we will add.

Seeing It Work

All of this is visible. Grit ships with an explorer, a single Python script that starts a local web server, indexes any codebase or clones one from GitHub, and exposes every layer of the harness on screen.

python explorer.py
# open http://localhost:8080
# paste https://github.com/psf/requests
# click Clone & Explore

The repo clones, tree-sitter parses every file, both graphs build, and the UI comes up with a search bar and five tabs. Here is what it shows for a real bug.

Take a genuine one from the requests library: Session.resolve_redirects copies the original request. Type that into the search bar and hit enter. The results come back ranked, each with its per signal scores, the function body visible inline with no clicking:

The harness found the bug site and the model never touched the codebase. The lexical signal caught "redirect" in both the path, sessions.py, where redirect logic naturally lives, and the symbol name, resolve_redirects. The definition boost pushed the real definition above the test files that reference it.

Search results for "Session.resolve_redirects copies the original request." The harness found the right function without the model touching the codebase.

Click a result and every other tab updates. Symbols lists every function and class in the file with signatures. AST shows the tree-sitter parse, byte ranges on every node. Edges filters to what the file imports and what calls into it. Model auto fills the selected function, ready for a test edit against any local Ollama model.

The Symbols tab lists every function and class in the file. Click any symbol to see its body and byte range.

The AST tab shows the tree-sitter parse with byte ranges on every node. These byte ranges are what make whole function replacement possible.

The whole thing is deterministic. No model was called for the search, the ranking, or the graph. The harness built the index, ran seven signals, ranked the candidates, and drew the result. The model tab is there for convenience. The vision layer does not need it.

The Model tab is ready when you need it. But the vision layer got here without calling it once.

What's Next

The vision layer is the first third of the harness. The next two pieces, feeding the code to the model and placing its edits back into the file, are where the real decisions live. SEARCH/REPLACE turns out to be the wrong format for a small model. Whole function editing with tree-sitter byte range swaps removes match failures entirely. A quality monitor catches bad output before it reaches disk. And there is a bug that cost me two days, where byte offsets from a systems tool refuse to line up with character indices from a high level language, which is a story about an em dash and a lesson about the seam between two levels of abstraction. Another article.

The point for now is smaller. Before a model can fix a bug, something has to find the right code. That something should be deterministic, local, and fast. It should work the same way every time, on any repo, without an API key or a GPU. The model's job is to edit. The harness's job is everything else. Nothing in the codebase acts without the harness having seen it first.

Grit is in active development, about 2,700 lines of Python across 20 modules. The indexer ships with the Python tree-sitter grammar today; the architecture is language agnostic but only Python is wired up.