Prashanth Pai (CodeRabbit) gave the practitioner's tour of running untrusted, model-generated code safely — at a scale most people never reach. CodeRabbit reviews 12M+ PRs and spawns ~500,000 sandboxes a day (peaking ~33k/hour), each living 3–10 minutes, starting in ~700ms. The threat model is concrete: an LLM running code can exfiltrate secrets, reach internal networks, poison builds, or burn money via crypto-mining and fork bombs. The defenses form a layered toolbox — Linux primitives (namespaces, seccomp, cgroups) → containment tools (nsjail, bubblewrap) → microVMs (Firecracker, gVisor, Kata) → and an egress proxy that brokers or tokenizes secrets so the sandbox never holds a real credential. CodeRabbit's own design keeps the agent harness outside the sandbox and streams commands in, with a credentials broker at the boundary.
This is the operational sibling of Zero Trust for Agents (WSO2): that talk set the identity/policy architecture; this one is the gritty execution-isolation layer underneath it. It also closes the loop with Day 1's agent talks (DD10/DD11).
Why sandbox at all?
A sandbox is an isolated environment designed for AI agents to safely run untrusted code. The use cases are everywhere now: tool calls, MCP servers, running LLM-generated code, running user-generated code, coding agents, and persisting/restoring state — on local laptops (Claude/Codex/OpenClaw) and on remote servers (background agents). When a model can write and execute code, "untrusted code" is the default, not the exception.
Where the harness lives — the key design choice
The first architectural decision: is the agent harness (the agent loop, MCP tools, filesystem) inside the sandbox or outside it? Drawing on OpenAI's sandboxes guidance, the talk laid out two patterns:
| Harness in compute | Harness separate from compute |
|---|---|
| The harness runs inside the sandbox (agent loop, MCPs/tools, filesystem). A gateway service holds secrets and intercepts calls so the sandbox can't make untrusted HTTP requests directly. | Secrets and the harness run in a trusted environment (anywhere — Temporal, AWS); the sandbox runs anywhere (E2B, Vercel…). The server accesses databases and the web from the trusted side, never from the untrusted sandbox. |
Sandboxes @ CodeRabbit — the second-gen design
CodeRabbit's current execution environment is a Linux microVM sandbox. The layering (top to bottom): a Cloud Run instance hosts the CodeRabbit "Sandboxer," which spawns multiple nsjail sandboxes over an overlayfs and a tokenizer proxy, all on a guest Linux kernel inside x86 virtualization (a VMM at host ring 3), with a syscall filter and Linux namespaces, on the host kernel.
The operational numbers are the headline:
| Metric | Value |
|---|---|
| Sandbox startup | ~700ms |
| Most command runs | ~250ms |
| Third-party tools available | 50+ |
| Sandboxes spawned | ~500,000 / day (peaks ~33k / hour) |
| Sandbox lifecycle | 3–10 minutes |
Autonomous agents — harness inside, stronger isolation
For fully autonomous coding agents, CodeRabbit flips to harness inside a Firecracker microVM, accepting more operational complexity for stronger isolation and dependency-install flexibility, single-tenant per VM. The agent lifecycle: a coding ask → durable workflows → a sandbox that can reuse/restore, fronted by a firewall + credentials broker, with a resident supervisor → coding agent → repo workspace, backed by command/event journals and an executor wake, going warm-idle then shutdown.
What makes up a sandbox — the toolbox
The middle of the talk was a genuinely useful taxonomy: "containers are legos." You assemble isolation from Linux primitives and containment tools.
| Layer | Pieces |
|---|---|
| Containment tools | nsjail, bubblewrap, minijail, firejail, httpjail, systemd-nspawn, runc, crun, sandstorm, island |
| Linux primitives | namespaces (mnt/pid/net/ipc/uts/user), cgroups (v1/v2), seccomp-bpf, capabilities, rlimits, mounts/bind-mounts, chroot/pivot_root, overlayfs, AppArmor/SELinux, Landlock/LSM |
Knowing what each primitive actually does is what lets you reason about a sandbox's strength, because "it's in a container" tells you almost nothing on its own — a container is just a curated bundle of these:
| Primitive | What it isolates / limits |
|---|---|
| namespaces | What the process can see: pid (other processes), net (interfaces/routes), mnt (filesystems), ipc, uts (hostname), and user (UID mapping — the one that lets "root in the sandbox" be an unprivileged UID on the host). |
| cgroups (v2) | What it can consume: CPU, memory, PIDs, I/O. This is the defence against the DoS/cost threat — a fork bomb or crypto-miner hits a hard ceiling instead of taking the node down. |
| seccomp-bpf | Which syscalls it may make. Blocking the long tail of rarely-needed syscalls shrinks the kernel attack surface — fewer entry points to exploit a kernel bug. |
| capabilities | Which slices of root power are granted (drop CAP_NET_RAW, CAP_SYS_ADMIN, etc.) — so "root" inside isn't full root. |
| LSMs (AppArmor / SELinux / Landlock) | Mandatory access control — fine-grained allow/deny on files, paths, and operations, enforced by the kernel regardless of UID. |
| chroot/pivot_root + overlayfs | The filesystem view: a disposable, often read-only rootfs the sandbox can't escape or persist beyond. |
MicroVMs — the strong tier
For real isolation you move to microVMs, which add a guest kernel and hardware virtualization under the workload. The defense-in-depth stack:
- Untrusted workload — AI-generated code, CI scripts, user repos, dependency hooks, tests — with no secrets, network policy, CPU/memory/time limits, no persistent data.
- Container runtime inside the guest — Podman, OCI images, no host Docker socket — rootless where possible, read-only mounts, dropped capabilities, seccomp/AppArmor.
- Ephemeral guest OS — dedicated kernel, disposable rootfs, no host credentials, minimal packages.
- MicroVM / VMM layer — Firecracker, crosvm, Cloud Hypervisor, libkrun, QEMU-microvm — minimal device model, seccomp filter, jailer, small TCB.
- Host OS & KVM — bare metal, hardened kernel, minimal services, worker isolation, IOMMU, audit logs — over hardware virtualization (Intel VT-x / AMD-V).
The isolation spectrum — pick your point
There's no single "secure" answer; there's a spectrum trading isolation strength against speed, compatibility, and overhead. The whole talk is really about choosing a point on it per workload:
| Approach | Boundary | Isolation | Start / overhead | Compatibility |
|---|---|---|---|---|
| Plain container | Shared host kernel (namespaces) | Weakest — one kernel bug escapes | Fastest, near-zero | Full |
| Hardened container | + seccomp, dropped caps, LSM, rootless | Better, still shared kernel | Fast | Full |
| gVisor | User-space kernel (Sentry) + host | Strong — two kernels to break | Fast start, some syscall overhead | Most apps; some syscalls unsupported |
| Kata / microVM | Own guest kernel + hardware virt | Strong — VM boundary | Heavier, ~100s ms+ | Near-full (real kernel) |
| Firecracker microVM | Minimal-device VM + small TCB | Strongest practical, tiny attack surface | ~100–700ms, low memory | Near-full |
CodeRabbit's two postures are just two points on this table: nsjail-inside-a-microVM for fast multi-tenant command runs (hardened-container speed plus a VM boundary), and Firecracker single-tenant for autonomous agents (max isolation, accept the cost). The art is matching the point to the trust and latency budget — you don't put a 700ms microVM in front of a call that must return in 50ms, and you don't run truly untrusted autonomous code in a plain shared-kernel container.
~/.aws/credentials or ~/.ssh/id_rsa and POSTs it out. Internal network: the classic is an SSRF to the cloud metadata endpoint (169.254.169.254) to steal the node's IAM role — which is why egress network policy and blocking link-local addresses is non-negotiable. Poison: a dependency post-install hook (npm/pip) runs arbitrary code at install time and backdoors the build or cache. DoS/cost: a fork bomb or crypto-miner pegs CPU and runs up the bill. Note how each maps to a specific defence: exfil → no secrets in the sandbox + egress proxy; internal network → network policy; poison → ephemeral, read-only, no persistence; DoS → cgroup limits + short lifecycle. The defences aren't generic "isolation" — they're targeted at named attacks.The Kubernetes-native path: Agent Sandbox
On Kubernetes specifically, the emerging pattern is an Agent Sandbox for executing isolated, low-latency tasks: an Agent Orchestrator pod calls an Executor (API/SDK) with init()/run()/read(); the Executor hits the K8s API and an Agent Sandbox Controller that allocates from a warm pool (and replenishes it); the task runs in a sandbox with a gVisor execution process, ephemeral storage, and a network policy. The warm pool is what gets you low latency despite the strong isolation.
The hard part — secrets without exposure
If the sandbox is untrusted, how does the code inside it ever call an authenticated API? It can't hold the credential. The talk's answer is an egress proxy, with two flavors.
Broker secrets
Fig 1 — broker pattern: the sandbox sends requests with no secrets; the egress proxy fetches the real credential from a secret store and injects it before forwarding. Example: Envoy + a credential injector, rules per sandbox.
The sandbox makes a request without secrets; the egress proxy (which alone has access to real credentials) injects them and forwards to the external service. You define injection rules per sandbox. The catch is that the proxy is stateful — it must hold and look up real credentials.
Tokenize secrets
The slicker variant: a credential issuer encrypts the secret with the proxy's public key, producing an opaque token that's injected into the sandbox. The sandbox sends "request + encrypted secret"; the egress proxy decrypts with its private key and injects the real credential. Now the proxy is stateless — the behavior is encoded in the payload itself — though format validation can fail. This is the superfly/tokenizer model.
The proxy toolbox
| Proxying tool | Role |
|---|---|
| Envoy + credential_injector | inject creds at egress |
| Mitmproxy | TLS-terminating interception |
| Squid | caching proxy with access-control lists |
| LiteLLM | gateway + rate limiting |
| superfly/tokenizer | tokenizing proxy |
Traffic gets redirected into the proxy via base-URL rewriting, HTTP_PROXY, proxychain, iptables, or httpjail.
Or just use MCP / custom tools
The cleanest option when it fits: have the agent call a tool (a service/proxy) that injects the credentials. Pros: no TLS interception needed, and credentials stay out — the agent only ever sees the tool interface. Cons: you host and maintain MCP servers, and MCP authorization is still required. (This is where it joins Day 2's gRPC-for-MCP story — the tool interface becomes the security boundary.)
FAQ
Why isn't a container enough to sandbox an LLM?
A plain container shares the host kernel via namespaces/cgroups, so a single kernel vulnerability can mean escape. For untrusted, model-generated code at scale, you add stronger layers — gVisor (a distinct user-space kernel) or a microVM (Firecracker/Kata, with its own guest kernel and hardware virtualization) — so an attacker must defeat two layers, not one.
Should the agent harness be inside or outside the sandbox?
It depends on trust. CodeRabbit keeps the harness outside for quick multi-tenant command execution (the brain and secrets stay out of the blast radius; the sandbox is a dumb executor). For long-running autonomous agents it moves the harness inside a single-tenant Firecracker microVM, accepting more operational complexity for stronger isolation.
How does sandboxed code call an authenticated API without holding the secret?
Via an egress proxy. In the broker pattern the sandbox sends a request with no secret and the proxy injects the real credential from a secret store. In the tokenize pattern the sandbox carries an opaque token (encrypted to the proxy's public key) and the proxy decrypts and injects — making the proxy stateless. Either way the sandbox never possesses a usable credential.
What scale are we talking about?
CodeRabbit spawns ~500,000 sandboxes per day (peaking ~33k/hour), each living 3–10 minutes, starting in ~700ms, with most commands running in ~250ms across 50+ tools. That scale is why warm pools and fast-start microVMs (Firecracker, nsjail-in-microVM) matter so much.
Is gVisor or a microVM "better"?
Different trade-offs. gVisor needs no hardware virtualization and has fewer kernel-invasive parts, intercepting syscalls in a user-space kernel. MicroVMs (Firecracker/Kata) give a full guest kernel with hardware virtualization and a small TCB. Many designs combine them — e.g. gVisor execution inside a Kubernetes Agent Sandbox, or nsjail inside a microVM as CodeRabbit does.
Broker or tokenize — which egress-secret pattern should I use?
Broker is simpler to start with but the proxy is stateful (it holds and looks up real credentials, so it's a high-value target and a scaling bottleneck). Tokenize makes the proxy stateless — the secret travels as an opaque token encrypted to the proxy's public key, so the proxy just decrypts and injects, and it scales horizontally with nothing to store. The cost is more moving parts (a credential issuer, key management) and that malformed/expired tokens fail at request time. For high scale, tokenize (the superfly/tokenizer model) is the cleaner fit; for a handful of sandboxes, broker is less to build.
Why is "no secrets in the sandbox" the load-bearing rule?
Because you should assume the sandbox will be compromised — that's the whole premise of running untrusted code. If the sandbox never holds a usable credential, then even a full compromise can't exfiltrate one; the worst case is bounded to whatever calls the egress proxy's per-sandbox rules already permit. Isolation (gVisor/microVM) reduces the chance of escape; keeping secrets out reduces the blast radius if it happens anyway. You want both — defence in depth assumes each layer can fail.
Why does CodeRabbit run nsjail inside a microVM rather than just one or the other?
Defence in depth with a speed budget. The microVM gives a hardware-virtualization boundary (escape the nsjail and you're still in a VM, not on the host). nsjail inside the VM cheaply isolates the many short-lived per-command sandboxes from each other within that VM, without paying a full VM boot per command. So the VM is the trust boundary against the host; nsjail is the cheap multi-tenant divider inside it — together they hit ~700ms starts at 500k/day while still having a real VM between untrusted code and the node.
Takeaways
- Model-generated code is untrusted by default — the threats are exfil, internal-network access, poisoning, and DoS/cost.
- Decide where the harness lives — outside (dumb executor, secrets safe) for multi-tenant speed; inside a single-tenant microVM for autonomous agents.
- Isolation is layered — Linux primitives → containment tools (nsjail/bubblewrap) → gVisor / microVMs (Firecracker/Kata); make attackers break two kernels.
- Secrets never enter the sandbox — an egress proxy brokers (stateful) or tokenizes (stateless) credentials; or call an MCP/tool that injects them.
- It works at scale — CodeRabbit runs ~500k sandboxes/day, ~700ms start, 3–10 min lifecycle, via nsjail-in-microVM + warm pools.
Next in Day 2 — Zero Trust for Fintech, building secure banking infrastructure with Cilium.
References
- KubeCon Mumbai 2026 — Day 2 index · the rest of Day 2
- gVisor · user-space kernel sandbox
- Firecracker · microVM with a small TCB
- Kata Containers · per-pod lightweight VMs
- Day 2 — Zero Trust for Agents · the identity side of the same problem