The paper's central idea is wonderfully simple: when someone asks for a behavior change, the hard first question is often not “how do I write the patch?” but “where does this behavior actually live?” A repository is arranged as files, modules, and functions. A request is phrased as an outcome: ask before deleting a file, carry environment variables into a command, retry one class of failure, or stop after a verified success. The Harness Handbook builds the missing bridge. It reorganizes a harness around what the system does, uses three levels of detail to lead a reader from system flow to source-backed units, and tracks state that crosses stages. The source code remains the authority. The handbook is a map, not a replacement landscape.
Imagine it is Monday morning and a product manager writes one sentence: “When the coding agent deletes a file, always ask the user first.” It sounds like a one-line policy change. You search for delete_file, find a tool definition, add an approval check, and open a pull request. The review looks tidy. Then a test reveals that patches can also delete files. A second path invokes a shell command. Headless mode has an auto-approval branch. A retry path bypasses the prompt because it assumes approval was already recorded. The behavior was not one function. It was a chain spread across tool schemas, policy evaluation, UI state, sandbox execution, fallbacks, and tests.
This is the problem that Harness Handbook: Making Evolving Agent Harnesses Readable, Navigable, and Editable names behavior localization. The paper was posted as arXiv v1 on July 14, 2026 by Ruhan Wang and nine co-authors. Its subject is agent harnesses, but the underlying problem will feel familiar to anyone who maintains a large system: people ask for changes in the language of behavior, while code is stored in the language of implementation. A good change depends on translating between the two without missing a quiet path.
First: what exactly is an agent harness?
A foundation model accepts an input and produces an output. It does not, by itself, decide which repository instructions to load, remember a session, expose tools, ask for approval, run a process, stream its output, recover from a timeout, or save what happened. The harness is the software layer that performs those jobs. It turns model output into controlled system behavior.
For a coding agent, the harness commonly owns at least six responsibilities. It builds the context sent to the model. It describes and dispatches tools. It stores conversation and task state. It controls the loop that alternates model calls and actions. It applies permission and sandbox rules. It connects the agent to files, terminals, plugins, remote services, and user interfaces. Two products using the same model can therefore behave very differently because their harnesses make different decisions.
This distinction matters because a request such as “make commands time out after two minutes” does not target model weights. It targets the runtime around the model. The timeout may be declared in a tool schema, defaulted in a request parser, passed through an execution session, capped by a backend, surfaced in an error message, and covered by several tests. Changing the harness safely means understanding that distributed path.
The mismatch: requests speak behavior, repositories speak structure
A repository tree answers a useful question: where are files stored? A symbol index answers another: where is this name defined or referenced? Neither automatically answers: which implementation sites jointly create this runtime behavior? That last question needs a mental model of execution, state, alternatives, and boundaries.
Consider a natural-language request: “Allow one shell command to receive temporary environment variables without changing later commands.” There may be no exact phrase “temporary environment variables” in the source. The relevant implementation can include a JSON schema, two mirrored tool descriptions, a request type, shell and unified-exec routes, the process-spawn merge point, serialization, and tests. The authors' project page uses a version of this example and reports that a single behavior-level request expands to 14 coordinated code updates across 10 files. A keyword search can find env thousands of times; the work is deciding which occurrences belong to the behavior chain.
The mismatch creates two opposite failure modes. Under-localization means the plan misses a required site. The patch works on the main path but fails in a mirror, fallback, platform branch, or rarely executed mode. Over-localization means the plan includes too much. The agent touches unrelated utilities because names look similar, increasing review cost and regression risk. A useful representation must improve both recall and precision: find all the right places while avoiding the wrong ones.
Behavior localization: the missing step before edit planning
The paper defines behavior localization as finding all code locations that implement the behavior described by a modification request. Notice the word all. Finding one plausible function is search. Finding the complete, relevant implementation boundary is localization.
Localization happens before the patch plan. First establish what stages participate, what state connects them, which units implement those stages, which nearby calls matter, and whether the referenced source is still current. Only then decide what to edit. This ordering is important. A detailed plan built on an incomplete location set is still an incomplete plan. Fluency in describing a change does not repair a missing execution path.
Humans solve this with experience, debugging, reading, and memory. A senior maintainer often knows that “approval” lives in three packages and that Windows has a separate path. A coding agent must reconstruct that knowledge under a context budget. It can search iteratively, but every irrelevant file consumes tokens and attention. Long context increases the amount of code that fits, yet it does not tell the agent which code belongs together. The paper's claim is not that search or long context is useless. It is that the missing organizing principle is runtime behavior.
Fig 1 — The repository contains the evidence, but it does not provide the behavior-to-code translation automatically.
The Harness Handbook in one sentence
A Harness Handbook is a behavior-centered manual generated from a harness codebase, with every low-level entry linked back to source. Instead of beginning with directories, it begins with the system's execution story. It then reveals more detail only when the reader needs it.
The representation has a document tree with three levels, called L1, L2, and L3, plus a complementary state-register view. L1 describes the whole system. L2 describes stages or behavior units. L3 describes source-backed implementation units. The state-register view records values and relationships that cross stage boundaries. Together they answer four questions in order: How does the system run? Which part owns this behavior? How does that part work? Where is the current code evidence?
The levels are not merely three amounts of summary. Each level performs a different navigation job. L1 prevents local details from losing their place in the full lifecycle. L2 narrows a question to coherent behavior units. L3 grounds those units in implementation. The register view catches dependencies that a neat stage hierarchy would otherwise hide.
L1: begin with the system, not the directory
L1 is the system overview. It explains architecture, execution model, major stages, global state flow, design principles, and entry and exit conditions. A reader should be able to follow one request from arrival to completion without knowing a filename.
The generated Codex example shows what this looks like at large scale. Its overview describes startup, environment and home-directory discovery, configuration, authentication, persistence, service connections, session routing, context assembly, model streaming, approvals, sandboxed tool execution, result collection, and shutdown. That story is much closer to how an engineer reasons about a failing agent than a list of crates. If a request concerns command approval, L1 shows where approval sits between model output and execution. If it concerns lost session state, L1 shows the persistence and conversation stages that might participate.
L1 also protects against tunnel vision. Suppose the visible bug appears in a terminal UI. The system overview may reveal that the UI only renders a decision produced by a policy stage. Editing the UI would change presentation, not behavior. Conversely, a backend-only plan may omit the state needed to display a pending approval. The overview helps the planner ask upstream and downstream questions before opening source.
L2: turn the lifecycle into behavior units
L2 decomposes the overall flow into stages or components. Each unit explains its purpose, responsibilities, inputs, outputs, dependencies, key state, and interactions. The paper uses the phrase component overview; the project page often calls these behavior units. Both names point to the same job: provide a coherent place to reason about what part of the system does.
A stage might be “tool execution, approval, and controlled operations.” Beneath it can sit narrower units for approval policy, guardian review, hooks, permission requests, sandbox command runtime, and execution backends. This arrangement makes relationships visible. The user request does not need to name execpolicy or a particular Rust module. It can match the behavior unit, then descend.
L2 is also where scope control begins. If the request changes how a timeout is passed to command execution, the planner can select command orchestration and execution-session units while excluding unrelated authentication or rendering stages. When shared state points elsewhere, those stages can be added deliberately. The result is not “search the whole repository, then guess.” It is “select the behavioral neighborhood, follow its dependencies, then inspect the relevant implementation.”
L3: deep detail with a live source locator
L3 is the unit deep dive. It explains triggers, inputs and outputs, detailed execution logic, state transitions, exceptions, implementation notes, source locations, and useful snippets. This is the level where a behavior claim becomes checkable.
An L3 entry can be a function or a file, depending on how the handbook was built. In function-as-leaf mode, an entry can cover a whole function or one or more contiguous regions when the same function serves several behavioral roles. In file-as-leaf mode, the entry represents a file. Either way, the locator is derived from static analysis, not invented by prose generation.
The authors use a strict grounding rule: the model may explain static facts, but it may not alter file paths, function identities, signatures, source ranges, or resolved call relations. An unresolved call is logged rather than assigned a guessed target. If a locator no longer matches the repository, that entry cannot remain active simply because its explanation still sounds reasonable. This is a crucial difference between a handbook and an ordinary architecture essay.
The state-register view: follow data that ignores module boundaries
Execution stages are useful, but real behavior often travels through shared state. An approval request may be created in one stage, stored in a conversation or session object, rendered by another stage, and read later by an executor. A stage-only hierarchy can make each component readable while still hiding the thread that joins them.
The complementary register view records cross-stage state relationships. A register is not necessarily a CPU-style register or one literal variable. It is a named piece of system state whose writers, readers, and transitions connect behavior units. Examples include the current conversation state, accumulated tool results, approval status, loop counter, active execution session, or retry budget.
When a question enters the handbook, the planner first selects stages by behavior and then follows registers to coupled stages. This is how a seemingly local request discovers a distant consumer. The register view is especially valuable for bugs with delayed symptoms: a value is constructed incorrectly early, but the failure appears several stages later. Search finds the symptom name. State tracing finds the causal chain.
Two rules make the map trustworthy
Rule 1: progressive disclosure
Do not give the agent every detail at once. Start with L1 and L2, select what matters, then open L3 entries only as needed. This reduces cognitive and token load while keeping local code tied to the overall flow. Progressive disclosure is not aggressive summarization. The detail still exists; navigation controls when it enters working context.
Rule 2: behavior-implementation alignment
Every active L3 locator must resolve against the current repository. If it cannot be revalidated, the entry is frozen and excluded from localization until refreshed. The handbook may guide the search, but source verification decides what enters the evidence set. This prevents an old manual from silently becoming authority over new code.
These rules solve different risks. Progressive disclosure limits irrelevant context. Alignment limits stale or fabricated context. One improves efficiency; the other protects correctness. A short, current path to code is more useful than a huge, impressive document whose anchors have drifted.
| Representation | Good at | Still leaves to the reader |
|---|---|---|
| File tree | Ownership, packaging, discovering directories | How a behavior crosses those directories |
| Symbol search | Definitions and explicit references | Synonyms, implicit flow, state coupling, missing mirrors |
| Semantic search | Finding conceptually similar code | Completeness and verified execution relationships |
| Repository summary | Orientation and compact context | Current source evidence for each behavioral claim |
| Harness Handbook | Behavior-first navigation tied to source | Final verification and the engineering decision itself |
A worked example: “Will it ask before deleting?”
Start at L1. Locate the part of the lifecycle where a model request becomes a tool action. The system view says that tool calls pass through permission and sandbox controls before execution. That establishes the surrounding flow.
Move to L2. Select the tool-execution and approval units. The L2 cards identify responsibilities: parse the tool request, evaluate policy, request user confirmation when required, carry the decision into execution, and return an error on rejection. Follow the approval-state register to the UI or session unit that records and returns the user's response. Follow the call relation toward the actual delete or patch backend.
Open L3 entries. Check the trigger, policy rule, user-response state, execution branch, and exception paths. A serious audit asks more than whether the normal delete_file route prompts. It checks whether shell deletion, patch deletion, auto-approval modes, headless execution, retries, and legacy paths reach an equivalent gate. Each candidate locator is then opened in the current repository. Entries whose code no longer supports the claim are discarded or frozen.
The result is an evidence chain, not a reassuring sentence. It may conclude that deletion is protected on the main path but that one fallback behaves differently. That is a useful answer. A handbook designed to expose uncertainty should prefer a bounded “not verified” to a broad claim based on the happy path.
How different people use the same map
A new maintainer uses L1 and L2 to learn the execution story before memorizing packages. The map gives unfamiliar names a place in the lifecycle. When the maintainer opens a file, they know why it matters and what feeds it.
An incident responder starts from the observed behavior, follows state and calls, and checks rare error paths. The handbook narrows the search without claiming the diagnosis. Logs and runtime traces still decide what actually happened.
A reviewer compares the proposed diff with the behavior boundary. If the plan changes a schema but not a mirrored execution path, the missing relationship is easier to notice. If it touches five unrelated stages, the reviewer can challenge the scope.
A coding agent uses the same path under a token budget. It reads the system and stage index, opens only selected detail pages, expands along call relations, verifies current source, and writes a plan whose edit blocks cite evidence. The representation is human-readable, but its navigation structure also acts as a control surface for an agent.
What the paper does not claim
The Handbook does not replace the repository. The paper repeatedly makes current source authoritative. It does not show that every final patch will compile or pass tests; its experiment evaluates localization and edit-plan quality, not full end-to-end execution. It does not prove that the generated behavior structure is complete for every language feature or architecture. Conservative failure handling records gaps rather than erasing them.
It also does not argue that all documentation should be generated. Product intent, historical decisions, operational runbooks, and human judgment do not appear automatically in a call graph. The Handbook targets a narrower but valuable problem: connect runtime behavior to implementation evidence so readers can navigate, audit, and plan changes more reliably.
Finally, the paper studies two open-source agent harnesses, Codex and Terminus-2, with 30 requests per harness. That is meaningful evidence, not universal proof. The idea is strongest where behavior is genuinely scattered and repeated navigation cost is high. A tiny service with five files may not need this machinery. A large, fast-changing harness with multiple execution modes is exactly the environment where the missing map becomes expensive.
A practical mental model to carry forward
Think of the repository as a city. Files are addresses. Search is a directory. A call graph is a road map. None tells you the complete route followed by “a package arrives, fails inspection, requests approval, gets rerouted, and is finally delivered.” That route is behavior. A Harness Handbook overlays the operational route on the physical city and keeps links back to each address.
The most important design choice is not the prose. It is the direction of travel: behavior question → system context → behavior units → implementation detail → current source evidence. This direction matches how change requests arrive and delays source expansion until the reader has a reason to open it.
Part 2 explains how the authors build this map without letting the language model invent the streets: deterministic static facts first, LLM-assisted organization second, and source validation before packaging. We will also examine why small and large repositories use different leaf modes, what the proposer-reviewer loop does, and how uncovered code remains visible.
A fifteen-minute exercise: map one behavior by hand
You do not need the generator to feel the value of the representation. Pick one behavior in a repository you know: “how does a command receive approval?”, “where is a retry budget consumed?”, or “what survives a process restart?” Write a one-paragraph L1 answer that traces the behavior through the full system. Avoid filenames. If you cannot describe the flow, local edits are premature.
Next, write three to seven L2 cards. Give each card a behavioral name, one responsibility, inputs, outputs, and the state it reads or writes. A card called “utilities” is a warning because it describes a directory rather than a role. A useful card sounds like “evaluate command permission,” “record the user's approval,” or “launch the controlled process.” Draw arrows for execution and a different kind of arrow for shared state.
Now create L3 evidence rows. For every card, record the current path, function or region, a short excerpt, and one sentence explaining why it implements the behavior. Mark every claim as verified, uncertain, or missing. Search for mirrors and negative paths: rejection, timeout, retry, headless operation, alternate platforms, and compatibility code. If a behavior claim has no source evidence, do not polish the prose—record the gap.
Finally, ask a colleague who does not know the repository to answer the original question using your map. Note where they need a directory search, where the stage name misleads them, and where a state relationship is absent. This tiny usability test exposes whether the map is truly behavior-centered. It also creates a baseline for automation: a generated Handbook should make this exercise faster and more complete, not merely produce more pages.
The exercise reveals why the paper's representation has four pieces. L1 preserves the story. L2 creates a manageable routing layer. L3 makes claims checkable. Registers preserve relationships that the tree cannot express cleanly. Remove any one and the reader either drowns in detail, loses the system, or cannot verify the explanation.
Repeat the exercise after a small code change. If the function moved, update the locator. If a new fallback appeared, decide which stage owns it and whether a register relationship changed. If you cannot update the map without rereading the whole repository, the representation lacks reusable structure. If you update the prose but cannot prove which source facts changed, it lacks grounding. This manual resynchronization is a miniature version of the pipeline covered in Part 3, and it makes documentation drift concrete: drift is not old wording alone; it is a broken path between a behavioral statement and current implementation.
Save the question, evidence rows, and corrected map as a small evaluation case. Over time, a collection of real behavior questions becomes more valuable than generic documentation screenshots. It shows which paths repeatedly confuse readers, which stage names survive refactors, and where static analysis needs help. Include at least one rejected candidate so precision is tested alongside coverage. The Handbook idea becomes an engineering practice when navigation quality can be tested against concrete questions, not judged only by how complete the landing page appears.
Takeaways
- An agent harness is the software that turns model output into stateful, tool-using, permission-aware action.
- Behavior localization means finding every implementation site that jointly produces a requested behavior.
- A file tree, symbol index, or long context helps inspection but does not automatically supply the behavior-to-code mapping.
- The Handbook uses L1 for the system, L2 for behavior units, L3 for source-backed details, and registers for cross-stage state.
- Progressive disclosure reduces irrelevant context. Source alignment prevents stale prose from becoming authority.
- The repository remains the source of truth. The handbook guides verification; it does not remove the need for verification.
Frequently asked questions
Is a Harness Handbook just generated documentation?
No. Ordinary generated docs often summarize files one by one. The Handbook reorganizes implementation around runtime behavior, adds cross-stage state relationships, provides progressive navigation, and requires active low-level locators to resolve against current source.
Why not give the coding agent the whole repository in a large context window?
More context increases capacity but does not guarantee relevance or completeness. The agent must still infer which scattered sites form one behavior. Progressive disclosure tries to spend context on an evidence path rather than on undirected inspection.
Does the handbook replace code search?
No. It complements search. The handbook selects behavioral neighborhoods and relationships; search and direct source reads still verify names, implementations, tests, and current details.
Can this idea work outside agent harnesses?
Probably, as an engineering pattern. Workflow engines, compilers, control planes, and distributed services also have behaviors spread across stages and state. The paper evaluates agent harnesses, so broader effectiveness should be tested rather than assumed.
What is the most important safety rule?
Never let generated prose outrank current source. A candidate location should enter the edit evidence only after its locator resolves and its current code still supports the behavioral claim.
References and further reading
This series distinguishes the authors' measured findings from practical interpretation. The paper and official artifacts below are the primary sources; the final source provides broader context for why harness code matters.
- Paper (arXiv 2607.13285v1) ↗
Wang et al., submitted July 14, 2026. The primary source for the representation, algorithms, experiment, and reported results. - Official project page ↗
The authors' visual explanation, live examples, Codex handbook, Terminus-2 handbook, and Studio walkthrough. - Official implementation ↗
Open-source generators, planner helper, resynchronization workflow, and setup instructions. - Generated Codex Handbook ↗
A large file-as-leaf example showing how the behavior map looks on a real production coding-agent repository. - Generated Terminus-2 Handbook ↗
A smaller function-as-leaf example with a supplied execution skeleton. - Code as Agent Harness ↗
Related survey that frames code as the operational substrate for reasoning, action, state, feedback, and coordination.