The construction pipeline follows a facts-first bargain. Static analysis owns names, locations, signatures, call edges, and observed state access. A language model helps arrange those facts into execution stages and explain them in plain language. The model may interpret; it may not rewrite the facts. Construction happens in three phases: extract a program graph, organize source units around behavior, then synthesize and validate the L1-L3 handbook. Small, well-understood codebases can use functions as leaves and begin from a trusted execution skeleton. Large repositories can use files as leaves and infer the stage skeleton bottom-up. In both modes, uncovered or uncertain material stays visible instead of being quietly guessed away.
Part 1 described the result: a reader starts with the whole system, narrows to a behavior unit, opens a source-backed detail page, and follows state across stages. That representation sounds neat on a slide. The hard engineering question is how to generate it from a messy repository without turning an LLM's confident summary into fake architecture.
The paper's answer is a division of labor. Machines that parse code are good at exact structural facts. Language models are good at naming patterns, grouping related responsibilities, and writing explanations. Harness Handbook combines them in that order. Deterministic facts constrain semantic organization. Later validation checks that the source links still exist. When either side is uncertain, the system records the gap.
The contract: prose can explain facts, not invent them
A generated handbook contains two kinds of information. Grounding facts include file paths, function identities, signatures, source spans, classes, resolved calls, and hashes. These must come from the repository and parser. Behavioral descriptions explain what a unit does, why it belongs to a stage, which state it touches, and how it participates in the lifecycle. These can be model-assisted, but they remain downstream of the facts.
This contract is stricter than “ask a model to summarize every file.” A summary-only system can omit inconvenient files, confuse similarly named functions, or describe an architecture that sounds plausible but is not traceable. Harness Handbook creates a program graph first. If a call target cannot be resolved, it is placed in an unresolved-call log. It is not assigned to a likely-looking function. If a source unit cannot be organized, it is recorded as unmapped or uncovered. It is not forced into the nearest stage just to make the diagram complete.
The constraint also improves review. A human can disagree with a stage label while still trusting that the linked signature and location are real. Semantic organization is revisable; extracted identity is auditable. That separation is the foundation for later resynchronization, because exact source facts can be compared across versions.
Choose the leaf before you build the tree
The lowest handbook level, L3, needs a stable unit. The paper supports two choices: function-as-leaf and file-as-leaf. The selected leaf mode remains fixed for the life of a handbook, including updates. This prevents the representation from changing its basic meaning every time code changes.
Function leaves give fine-grained localization. A planner can land directly on a function or even a contiguous region within a function. That precision is valuable in a smaller harness whose execution stages are already known. The cost is scale: tens of thousands of functions create a large classification and narration workload, and a trustworthy stage skeleton is needed to guide the mapping.
File leaves trade granularity for coverage and scale. Every scanned file gets a card, the system infers an execution-stage skeleton from those cards and the program graph, and files are assigned to stages. The final planner still opens current source before proposing edits, so a file leaf is a routing unit rather than permission to edit the whole file.
| Decision | Function-as-leaf | File-as-leaf |
|---|---|---|
| Best fit | Smaller harness, reliable lifecycle skeleton, function-level budget | Large repository, no trusted skeleton, coverage is the first priority |
| Starting point | Seed skeleton supplied by the builder | Stage skeleton inferred from file cards and graph facts |
| L3 unit | Whole function or contiguous source regions | One source file |
| Primary advantage | Precise symbol-level behavior mapping | Scales bottom-up and keeps every scanned file visible |
| Primary risk | Bad seed skeleton distorts organization; classification cost can grow | A file may contain several behaviors; later source inspection must narrow it |
| Paper example | Terminus-2 | Codex |
Phase I: extract a program graph without an LLM
Phase I parses the repository through language-specific adapters. The official implementation states support for Python, Rust, TypeScript, Go, plus Starlark, Shell, and PowerShell, with automatic language detection for mixed repositories. The paper describes the common facts: functions or methods, classes and modules, signatures, line ranges, calls, named external boundaries, and observed state reads and writes.
An internal function becomes a graph node with a qualified identity and source locator. A known external target can become a named boundary node. Calls become edges only when the target resolves to one of those recognized nodes. This “resolved or recorded as unresolved” policy matters because false edges are especially dangerous in a behavior map: one invented relationship can pull an unrelated stage into the proposed edit boundary.
Phase I makes no language-model calls. The result is a program graph that later phases can organize but not falsify. A builder can run this phase by itself as a smoke test. The official large-repository pipeline documents a phase-1 command that points at a source root and work directory. This is a sensible adoption path: validate parsing and coverage before spending model tokens or trusting generated prose.
cd handbook_generate_large # Static facts only. No LLM required. python3 run.py --lang auto --source-root /path/to/repository --work-dir work/repository --phase 1
This command comes from the project's README, not from the experiment protocol. The implementation is evolving software, so flags should be checked against the current repository before use. The durable idea is the order: prove that the parser sees your code, inspect unresolved facts and coverage, then build the semantic layer.
Phase II in function mode: map code into a trusted skeleton
Function-as-leaf mode starts with a seed skeleton that defines execution stages and state registers. For Terminus-2, a concise lifecycle might include configuration, environment preparation, run initialization, the observe-think-act loop, completion confirmation, and cleanup. The skeleton does not assign functions; it provides the behavioral containers into which functions can be placed.
The organizer considers each analyzable internal function using its source, callers, callees, neighboring assignments, and stage descriptions. A function can belong to more than one stage. If different parts of a long function perform distinct roles, the pipeline can assign contiguous regions separately. This is important in real orchestrators, where one large function may load state, call a model, dispatch a command, update history, and decide whether to stop.
Initial assignments are proposals, not final truth. Later rounds review both membership and the skeleton itself. A stage can be added, removed, merged, or split when evidence shows that the initial structure does not fit. Structural changes must preserve a valid hierarchy and valid register references. Affected functions are reclassified against the revised structure.
The process stops when the skeleton and assignments stabilize or when its budget is exhausted. Budget exhaustion does not turn remaining uncertainty into certainty. Unmapped functions remain recorded. This is a subtle but strong design choice: coverage metadata is part of the product, not an embarrassing build log to hide.
Regions, boundaries, and the danger of long functions
Assigning a whole function to a stage is simple when the function has one responsibility. Orchestrator functions often do not. A single loop body can observe a terminal, compact memory, invoke a model, parse its response, execute commands, and check completion. Mapping that function wholesale to five stages makes localization noisy. Splitting arbitrary lines is also unsafe because a region could begin in the middle of a control structure.
The function pipeline therefore aligns retained regions to parser-derived statement spans where possible and stores source hashes for later comparison. Region actions must use legal ranges and cannot conflict. A boundary that cannot be aligned is marked for review. Later validation decides whether the locator remains active or must be frozen.
After classification, deterministic cleanup removes redundant whole-function entries when regions already cover the same role. Cross-stage references are rebuilt, unmapped functions are recorded, and entries within each stage are ordered using purposes and stage-internal calls. If a proposed order is incomplete or contains duplicates, file and line order provides a stable fallback.
Phase II in file mode: describe everything, then infer stages
File-as-leaf mode has no seed skeleton. It begins by producing a card for every scanned file, including files from which static analysis extracted no function. A card describes the file's purpose, repository role, and place in the execution lifecycle. In deep mode it also contains a static inventory of function identities, signatures, source ranges, and resolved calls. The model can explain that inventory but cannot edit it.
The pipeline uses a fallback sequence so a failed batch does not silently remove a file. An omitted file is retried by itself. If deep card generation still fails, the source can be retried in function-range chunks. If prose still cannot be produced, a fallback card is emitted. In deep mode, that fallback retains the graph-derived inventory. The file remains available to later phases and is marked undescribed in the coverage record.
Next, cards are grouped by directory and combined with entry points and resolved calls. From this context, the organizer drafts an ordered stage skeleton and assigns every scanned file to one primary stage when possible. Cross-cutting utilities such as logging, protocol definitions, types, or configuration can name one or two secondary stages. Missing assignments and unknown stage names remain visible rather than being coerced.
The paper describes several organization variants. A one-shot mode makes one stage draft and one assignment pass. A doctor-style mode refines the draft through reviewed iterations. An agent-produced draft can also enter that refinement path, with fallback to one-shot if the draft is invalid. Refinement can add, remove, merge, or split stages, then reconsider only affected or unassigned files. It stops when coverage and structure stabilize, improvement stalls, or the budget is reached.
Organize within stages so navigation feels like a story
A bag of correctly assigned files is not yet a readable manual. Within each stage, the file-call graph provides an initial caller-before-callee order. File cards then help create thematic groups and a sequence that a reader can follow.
Deterministic checks remove unknown or duplicate paths and ensure that every assigned file appears exactly once. Omitted files are appended to a fallback group. If the organization call fails, the pipeline uses a deterministic flat order. This is another example of graceful degradation: a less elegant handbook is better than a polished handbook that quietly loses code.
The official Codex handbook illustrates the scale this mode targets. The project page describes thousands of files, tens of thousands of functions, and a very large connection graph. Its generated index moves from process startup and environment discovery through configuration, identity, persistence, transport, model communication, rendering, tool execution, approvals, and sandbox runtimes. The stage map is not proof that every categorization is perfect. It is a navigable first hypothesis whose leaves still point back to source.
Fig 1 — Facts flow forward; uncertainty becomes audit data rather than disappearing behind fluent prose.
Phase III: synthesize the three-level handbook
Phase III converts the stage skeleton and organized source into the document tree, register view, and rendered package. Function and file modes reach the same reader-facing structure from opposite directions.
Function mode builds top-down. The known skeleton supplies the outline. The system generates the L1 overview, each L2 stage overview, then each stage's L3 function entries. Multiple retained regions from one function can be merged into a coherent entry. Each node uses a bounded generate-review-revise process: retain the best draft and stop when it passes the rubric, stops improving, or reaches the budget.
File mode builds bottom-up. File cards become L3 entries. A stage is summarized when it owns files or has child stages; empty leaf placeholders can be skipped. Parent summaries wait for child summaries, and the top-level stage summaries produce L1. This direction fits a repository whose lifecycle was inferred from source rather than supplied in advance.
The register view also differs by mode. Function mode begins from register declarations in the trusted skeleton; generation explains them without inventing identities. File mode proposes cross-stage registers from top-level stage summaries and data-model file purposes, repeating until no new useful registers appear or the budget is reached. References to missing stages are removed.
Ground every leaf before publication
Before rendering, the pipeline resolves every L3 locator against the repository and compares located source with stored evidence. An entry that fails revalidation is marked frozen and excluded from localization. This is the last gate between an attractive document and an operational navigation aid.
Why freeze instead of delete? Deletion hides that knowledge once existed and has drifted. Freezing preserves the explanation for review while preventing a planner from treating it as current evidence. An engineer can inspect the diff, repair the locator or mapping, and refresh the entry. The failure becomes actionable maintenance data.
The package stores more than rendered HTML or Markdown. It retains the leaf mode, program graph, current stage skeleton, mode-specific organization state, construction configuration, and reusable generation cache. Function mode also retains the original seed skeleton. This machine-readable synchronization state is what lets a later source diff refresh affected parts instead of rebuilding everything.
What the official repository adds to the paper
The paper presents the method and evaluation. The official GitHub repository exposes runnable generators and a planner helper. The README separates a large bottom-up pipeline from a small skeleton-driven pipeline. It documents deep per-file reading, doctor-style synthesis, optional HTML generation, English and Chinese narration, and phase-specific execution.
The large pipeline writes an overview, a per-stage index, a state-register page, individual stage pages, and an optional HTML site. The small pipeline writes Markdown and JSON from a supplied skeleton. The helper can package a rendered handbook as an agent skill, then run a single read-only planner that uses handbook navigation and opens real source before emitting an edit plan.
These artifacts make the research easier to inspect, but they also reveal practical cost. A full large-repository command can request deep reads and many parallel workers. Every non-static phase needs an OpenAI-compatible model endpoint. Generation is therefore an engineering pipeline with compute, retry, coverage, and maintenance concerns—not a free documentation button.
# Small repository: supply the lifecycle you trust. cd handbook_generate_small python3 run.py --lang auto --source-root /path/to/repository --skeleton skeletons/repository.yaml --work-dir work/repository --title "Repository Handbook" --project-name "Repository" --project-kind "coding agent" --project-brief "A terminal agent that edits code and runs commands." --out-lang en
Do not copy this command into production blindly. Pin the upstream revision, read the current README, use a non-secret configuration path, and begin on a disposable checkout. The code is research software and the paper is a July 2026 preprint. A safe pilot proves parser coverage, output quality, token cost, and update behavior on your own repository.
A practical quality checklist for generated handbooks
- Coverage: every scanned file or analyzable function is assigned, explicitly unmapped, or explicitly undescribed. There is no silent remainder.
- Identity: paths, signatures, and source spans come from static facts and resolve on the pinned repository revision.
- Call integrity: unresolved calls remain unresolved; reviewers can inspect the log instead of trusting guessed edges.
- Behavioral fit: stage names describe runtime responsibilities, not directory names with nicer wording.
- State flow: important cross-stage values have identifiable writers, readers, and transitions.
- Failure visibility: fallbacks, frozen locators, unmapped units, and budget exhaustion appear in a coverage report.
- Reader test: a new maintainer can answer a behavior question by moving L1 → L2 → L3 without knowing the original file names.
- Source test: every candidate edit location can be opened in current source and independently justified.
Where the construction can still go wrong
Static analysis has limits. Dynamic dispatch, macros, code generation, reflection, configuration-driven calls, foreign-function boundaries, and runtime plugin loading can hide relationships. Language adapters differ in maturity. A resolved call graph is not a runtime trace. The coverage record should tell reviewers what the parser saw, but it cannot turn unsupported semantics into facts.
Behavioral organization can also be wrong. A supplied skeleton may encode a maintainer's outdated mental model. An inferred skeleton may group files by superficial similarity. A model may write a good description of a bad assignment. Proposer-reviewer loops reduce this risk; they do not abolish it. Sampling source-backed stage paths and comparing them with runtime traces is a strong validation practice.
Finally, a handbook can be too detailed to navigate or too abstract to localize. Function mode may explode in size; file mode may be broad. The right leaf depends on repository scale, stage clarity, and the cost of opening current source. The paper offers two modes rather than pretending one representation fits every codebase.
Build a tiny handbook before scaling the generator
A useful pilot can begin with one request path rather than the whole repository. Choose an entry point, such as a CLI command that launches an agent turn. Run static extraction and inspect the graph around that entry point. Create a small seed skeleton with initialization, context assembly, model call, tool dispatch, state update, and termination. Then map only the functions that participate in one real trace.
Compare three views. First, the source tree: does it reveal the runtime order? Second, the graph: does it capture calls that matter, and where does dynamic behavior break the chain? Third, the handbook: can a new reader understand the path and open evidence without being told the filenames? This comparison tells you whether the main bottleneck is structural extraction, behavioral organization, or prose.
Deliberately introduce three failures. Rename a mapped function without changing its body. Add a new helper to the path. Change one call to a dynamically selected plugin. Re-run or resynchronize and inspect what happens. A healthy system shifts the renamed locator, classifies the new helper or marks it unmapped, and exposes uncertainty around the dynamic plugin. A dangerous system keeps a stale link, silently omits the helper, or invents a target.
Price the run. Record parser time, model calls, input and output tokens, retries, generated artifact size, manual corrections, and the number of source units that fall back. These numbers turn “the handbook looks impressive” into an engineering decision. On a large repository, coverage and resynchronization economics matter as much as the first render.
Keep the pilot's artifacts. The seed skeleton, parser coverage report, unresolved-call log, sampled review notes, and failure cases form an acceptance suite for future versions. If a new generator produces more fluent prose but loses a formerly visible gap, it regressed. The quality bar is navigable, grounded behavior—not literary smoothness.
Also test the handbook from two directions. In the forward test, choose an L1 stage and follow it down until every L3 locator opens the expected source. In the reverse test, sample source files or functions and ask where they appear in the behavior map. Forward-only review catches broken explanations; reverse review catches silent omissions. For cross-cutting utilities, confirm that the primary and secondary stage assignments explain the real role rather than duplicating the unit everywhere. A good map is neither a perfect tree nor a flat index. It is a controlled simplification whose exceptions remain visible.
Finally, ask the builder to state its confidence through evidence rather than scores alone. Which files used fallback cards? Which regions could not align to parser statements? Which stage assignments changed during review? Which registers were removed because their stages did not exist? These audit trails help a maintainer focus scarce review time on weak parts of the generated structure instead of rereading every polished paragraph equally.
Do one privacy review before the full run. The generator may send source chunks, file cards, stage summaries, and revision prompts to a configured model endpoint. Identify secret fixtures, customer data, proprietary subtrees, and generated artifacts that should not leave the approved boundary. Confirm retention and logging settings. Excluding a sensitive path must appear in coverage metadata, because “not scanned by policy” is different from “scanned and mapped.” A handbook that hides its exclusions can create false confidence even when the exclusions themselves are correct.
When the pilot passes, pin the generator revision and configuration beside the handbook snapshot. Record the source commit, language adapters, leaf mode, skeleton version, model, budgets, and known gaps. Reproducibility is part of grounding: readers should know not only which code the manual describes, but how that description was produced.
Takeaways
- Build exact program facts before asking a model to organize or explain code.
- Use function leaves when a reliable lifecycle skeleton and fine-grained budget exist; use file leaves when scale and coverage dominate.
- Let generation fail visibly. Retry, fall back, and mark gaps instead of dropping source units.
- Function mode organizes top-down from a seed; file mode organizes bottom-up from complete file cards.
- Validate every L3 locator against source before allowing it to guide localization.
- Package synchronization state, not only prose, because the codebase will change.
Frequently asked questions
Why does the leaf mode stay fixed?
A stable leaf contract makes links, caches, alignment, and resynchronization comparable over time. Switching from functions to files midstream would change the meaning of L3 entries and make incremental updates much harder to reason about.
Is function-as-leaf always more accurate?
No. It is more granular, but only if the parser, skeleton, and classification budget support that scale. A complete file-level map plus current-source verification can be more reliable than an incomplete function map.
Can I build Phase I without sending code to an LLM?
Yes. The paper states that static fact extraction makes no LLM calls, and the official implementation documents phase-1-only commands. Later organization and narration phases use an OpenAI-compatible endpoint.
What happens to code that cannot be classified?
It remains explicit in an unmapped or coverage record. Existing entries that can no longer be validated are frozen and excluded from localization until refreshed.
Should I trust an inferred stage skeleton?
Treat it as a source-backed navigation hypothesis. Review representative paths, compare with architecture knowledge and runtime traces, inspect uncovered units, and keep source verification mandatory for every edit plan.
References and further reading
This series distinguishes the authors' measured findings from practical interpretation. The paper and official artifacts below are the primary sources; the final source provides broader context for why harness code matters.
- Paper (arXiv 2607.13285v1) ↗
Wang et al., submitted July 14, 2026. The primary source for the representation, algorithms, experiment, and reported results. - Official project page ↗
The authors' visual explanation, live examples, Codex handbook, Terminus-2 handbook, and Studio walkthrough. - Official implementation ↗
Open-source generators, planner helper, resynchronization workflow, and setup instructions. - Generated Codex Handbook ↗
A large file-as-leaf example showing how the behavior map looks on a real production coding-agent repository. - Generated Terminus-2 Handbook ↗
A smaller function-as-leaf example with a supplied execution skeleton. - Code as Agent Harness ↗
Related survey that frames code as the operational substrate for reasoning, action, state, feedback, and coordination.