Jul 16, 2026 · ml · 13 min read · 2502 words intermediate

Reading the Grok Build Rust Workspace.

mlagentsgrok-buildharness-engineering

THE INCIDENT · CHAPTER 02

Mira clones the repository and opens the root Cargo workspace. Dozens of crates stare back at her. Reading them alphabetically feels like studying a city by memorizing every street name. She needs to know where a request enters, where decisions happen, and where side effects leave the process.

The question: How do you turn a large Rust workspace into a small mental map?

Start from first principles

A railway map omits buildings and trees. It keeps stations, lines, and transfers because those explain movement. A useful crate map does the same: it keeps runtime responsibilities and the boundaries between them.

Large Rust workspaces encourage directory tourism: open every manifest, restate its description, and mistake coverage for understanding. Grok Build has enough crates to make that approach actively misleading.

The better route starts at the binary, follows imports into a user-visible mode, and traces one prompt across ownership boundaries. Supporting formatting and protocol crates then make sense because we know which runtime path consumes them.

The root Cargo configuration is generated and marked read-only. Treat member manifests and source imports as the reliable map, and avoid interpreting generated workspace order as product priority.

In one sentence. A crate list is not an architecture. Read Grok Build from the composition root through client, runtime, action, state, and cross-cutting boundaries. The useful question is not 'what does this crate contain?' but 'which runtime contract becomes unstable if this crate changes?'
1 · NEEDHow do you turn a large Rust workspace into a small mental map?
2 · MECHANISMThe harness must own a clear harness-architecture boundary.
3 · PROOFObserve the model, harness, environment, and verifier separately.
The equation for the whole series. Coding-agent effectiveness = model capability × harness quality × environment quality × verification quality. If any factor approaches zero, the product approaches zero too. This chapter isolates harness-architecture, then reconnects it to the complete system.

Build the smallest useful mental model

Group crates into five bands: composition, clients, runtime, action/workspace, and cross-cutting services. This is a runtime map, not a dependency graph; a low-level crate can affect every layer without being a user-facing feature.

Rust boundaries matter where they constrain authority. A tool schema type should not execute a process. A client should not invent session semantics. A workspace proxy should not require the model loop to care about transport placement.

The purpose of the map is diagnostic. When headless output is wrong, start in the headless projector. When tool authorization is wrong, inspect the shell/permission path. When rewind misses a file, start in prompt-level file tracking rather than the pager.

Composition pager-bin Clients pager / ACP Runtime shell / agent Actions tools / workspace Services memory / MCP / sandbox cross-cutting crates supply contracts to every band feedback changes the next turn

Fig 2.1 — Responsibility bands in the Grok Build workspace.

Now open the hood

Only after the idea is clear does Mira open the source. She ignores most of the workspace and follows the few boundaries that must exist for this part of the story to work.

1. The next clue — Start at the composition root

Mira now needs one small mechanism: The executable must assemble modes while keeping feature implementations outside CLI parsing.

She follows that responsibility into the repository. pager-bin/src/main.rs imports run_headless, run_stdio_agent, and run_leader and dispatches through run_agent_command. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. Imports reveal the actual wiring better than crate names; they identify which subsystem owns process lifetime.

Then she tests the unhappy path: A flag can be accepted but ignored in a particular mode, so trace it from parser field into the called runtime rather than trusting help text alone. If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: crates/codegen/xai-grok-pager-bin/src/main.rs. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

2. The next clue — Separate clients from semantics

Mira now needs one small mechanism: Interactive rendering and headless projection should consume shared events instead of reimplementing the agent loop.

She follows that responsibility into the repository. The pager owns terminal presentation; headless.rs acts as an ACP client and the shell exposes stdio/server modes. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. This permits a rich TUI, scripts, and editors to share sessions and tool behavior.

Then she tests the unhappy path: Client-specific buffering or output projection can lose updates even when the runtime is correct; test event-to-output conversion independently. If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: xai-grok-pager, xai-grok-shell/src/agent/app.rs. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

3. The next clue — Put turn semantics in shell

Mira now needs one small mechanism: One runtime must own prompt lifecycle, tool feedback, compaction, cancellation, and stopping.

She follows that responsibility into the repository. The ACP session implementation under xai-grok-shell contains handle_prompt, recovery wrappers, and the conversation loop. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. Central ownership makes interface changes less likely to fork semantic behavior.

Then she tests the unhappy path: If a tool or client bypasses chat-state updates, later rounds reason from incomplete observations and persisted resume state diverges. If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: xai-grok-shell/src/session/acp_session_impl. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

4. The next clue — Keep agent definition distinct from session execution

Mira now needs one small mechanism: Prompt bodies, agent roles, tool selections, and discovered rules must be configurable without moving turn control into configuration code.

She follows that responsibility into the repository. xai-grok-agent provides builders, definitions, PromptContext, and layered AGENTS discovery consumed by the shell. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. It lets main sessions and subagents render different prompts/tools while using the same loop.

Then she tests the unhappy path: Treating the system prompt as a static string hides runtime-selected skills, audience, cwd, memory, and instruction precedence. If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: crates/codegen/xai-grok-agent/src/prompt/context.rs and agents_md.rs. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

5. The next clue — Treat chat state as a subsystem

Mira now needs one small mechanism: Messages, tool observations, request construction, and compaction metadata need an explicit state API.

She follows that responsibility into the repository. xai-chat-state is called by the turn implementation when user input, model output, and tool results enter the conversation. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. The model sees a projection of state, while persistence and UI may need richer events.

Then she tests the unhappy path: Mutating only a rendered transcript does not update the next model request; state ownership must remain unambiguous. If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: crates/codegen/xai-chat-state and its call sites in turn.rs. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

6. The next clue — Separate sampling from orchestration

Mira now needs one small mechanism: Provider/model streaming should return structured responses without owning tool execution or permission decisions.

She follows that responsibility into the repository. run_turn_via_sampler delegates sampling through xai-grok-sampler; the shell interprets calls and controls retries/compaction. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. Model transport can change without granting a provider adapter filesystem authority.

Then she tests the unhappy path: Authentication refresh and context overflow are transport/context recovery, not proof the engineering task recovered. If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: xai-grok-shell/.../turn.rs and xai-grok-sampler. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

7. The next clue — Tools define actions; workspace places them

Mira now needs one small mechanism: Tool registry code should describe and resolve operations, while workspace code chooses local/proxy execution and owns environmental state.

She follows that responsibility into the repository. xai-grok-tools defines ToolDefinition/FinalizedToolset; xai-grok-workspace binds sessions and dispatches call_tool. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. The boundary supports multiple placements and makes side-effect authority reviewable.

Then she tests the unhappy path: Collapsing both layers makes it difficult to distinguish 'tool absent' from 'workspace unavailable' or 'policy denied.' If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: xai-grok-tools and xai-grok-workspace/src/workspace_ops.rs. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

8. The next clue — Cross-cutting crates are architectural

Mira now needs one small mechanism: Memory, MCP, hooks, sandbox, config, telemetry, Markdown, and ACP must integrate through explicit contracts rather than scattered conditionals.

She follows that responsibility into the repository. Dedicated crates expose these services, while the shell and workspace consume them at defined lifecycle points. The important point is not the Rust syntax. It is ownership: this is where the system decides what crosses the boundary.

Why the story changes here. Cross-cutting does not mean optional trivia; these systems change context, authority, transport, recovery, and human comprehension.

Then she tests the unhappy path: A feature can be present in the workspace but disabled by configuration or absent from an effective toolset; installed is not the same as active. If the model, operator, and saved session do not receive the same honest outcome, the mechanism is not yet trustworthy.

Source: xai-grok-memory, xai-grok-mcp, xai-grok-hooks, xai-grok-sandbox, ACP crates. Verified against Grok Build c68e39f60462f28d9be5e683d9cbe2c57b1a5027.

Mira runs the experiment — trace one flag instead of reading seventy manifests

Reading source gives her a hypothesis. A small experiment tells her whether that hypothesis survives contact with a real workspace. Use --output-format as a vertical slice from CLI input to observable behavior.

  1. Find the parser field in pager-bin.
  2. Locate the headless-only validation and mode dispatch.
  3. Follow the value into the headless output projector.
  4. Identify plain, JSON, and streaming JSON branches.
  5. Trace session/update events that feed the projector.
  6. Confirm terminal metadata and spend caveats in the guide/source.
  7. Run a prompt with each format in a disposable repository.
  8. Compare stdout, stderr, exit status, and session ID.
rg -n 'output.format|OutputFormat|streaming.json' \
  crates/codegen/xai-grok-pager-bin \
  crates/codegen/xai-grok-pager

What she learns. The exact search is a source-reading technique, not a product command. It replaces crate enumeration with a testable cross-boundary contract.

The proof she demands. The value should resolve through parser, dispatch, runtime events, and final output without an unexplained duplicate implementation.

That last check matters. Grok Build can expose a mechanism and report an observation; the repository, operating system, CI platform, and reviewer decide whether those observations prove the actual task succeeded.

The whiteboard test

Before Mira explains the chapter to her team, she reduces it to three questions: what owns the decision, what evidence comes back, and what changes when the mechanism fails?

Review questionSource-backed answerOperational consequence
Where is process composition?pager-binKeep mode wiring out of tool implementations.
Where is turn state?shell plus chat-stateDebug semantic divergence here, not in CSS/rendering.
Where are side effects placed?tools resolved into workspace local/proxy operationsLog both operation and placement.
Where is human comprehension built?pager plus formatting/Markdown/Mermaid componentsTreat rendering errors as control-plane defects when they hide approvals or failures.

This is not a feature scorecard. A mechanism can work exactly as implemented and still be the wrong control for a particular threat. Defaults also change, so recheck the pinned source path before copying configuration into production.

Signals Mira keeps

  • Selected mode and resolved effective configuration.
  • ACP lifecycle/version and session identifiers.
  • Agent definition, model, toolset, workspace placement, and sandbox profile.
  • Per-crate error boundaries in traces rather than a single generic failure.

Together, those signals tell a complete story: the model proposed an action, the harness admitted and routed it, the environment performed something, and a verifier measured the result.

Limits and uncertainty

Generated root. The root Cargo configuration is generated; do not infer manual architectural intent from member ordering.
Dependency graph. A compile-time edge does not prove runtime ownership. Confirm call sites and state transitions.
Monorepo snapshot. Public sync boundaries can preserve internal naming that is not a public product concept.

The public repository is unusually detailed, but it is not the complete deployed product. The snapshot has one visible public commit, so it cannot support a rich historical explanation of why every boundary evolved. Hosted xAI model serving, account systems, and production topology remain outside this study. Where code and guide differ, this series gives the pinned implementation priority and marks documentation-only behavior instead of silently merging versions.

FAQ

Do I need to understand every crate?

No. Start with vertical runtime flows, then open supporting crates when a contract crosses into them.

Why is the shell not just a terminal wrapper?

It owns the ACP session and model/tool turn state. Unix command execution is one capability inside that larger runtime.

Why keep pager and shell separate?

The pager is a client/presentation surface; the shell exposes reusable session semantics to several clients.

Are formatting crates architecturally important?

They do not grant model capability, but they determine whether humans can inspect and control a long-running agent accurately.

What Rust knowledge matters most?

Understand enums/traits that select local versus proxy behavior, async task boundaries, and shared state ownership. Generic Rust syntax is secondary.

What changed for Mira

The workspace becomes five understandable neighborhoods: clients, runtime, actions, state, and cross-cutting services.

Next: With the map drawn, Mira can follow one request as it moves through the runtime.

Key takeaways

  • Read the workspace as runtime responsibility bands.
  • Start from composition and follow one user-visible value vertically.
  • Client, turn state, sampling, tools, and workspace have deliberately different authority.
  • Cross-cutting services alter core behavior even when they are not entry points.
  • A useful architecture map predicts where a failure should be debugged.

References & source notes

Freshness boundary. Grok Build claims in this article are pinned to c68e39f60462f28d9be5e683d9cbe2c57b1a5027. Pi comparison claims, where present, are pinned to 97f9978fa66685f78d2da19ae22e20c46d125f74; Hermes claims are pinned to c9c9bb33fcc6ab479846a1c496a6e9efe2c1c7d4. Recheck paths, symbols, commands, and defaults if those branches advance.

← Article 1 — SystemArticle 3 — Runtime loop →
© cvam — written in plaintext, served warm