← BPF Performance Tools

BOOK NOTES · BPF PERFORMANCE TOOLS · CHAPTER 5

BPF Performance Tools Chapter 5 — bpftrace.

bpf-performance-toolschapter-5bpftraceone-linersmapstracing

// the one-minute version

bpftrace is an awk-like language for writing custom BPF tracers in one line. Structure: probe /filter/ { action }where to attach, an optional condition, and what to do. Attach via probes (kprobe:, tracepoint:, uprobe:, profile:). Read context via builtins (pid, comm, nsecs, arg0, retval, kstack). Aggregate in-kernel with maps (@name) and functions like count(), hist(), sum(). BEGIN/END run setup/teardown. Learn this shape and you can answer almost any custom question in one line.

BCC gives you finished tools; bpftrace gives you a pen. When the question is bespoke — "what's the latency distribution of this specific function, filtered to that process, bucketed my way" — bpftrace answers it in a single line you type and discard. The language is deliberately small and awk-like, so the whole thing fits in your head. This chapter teaches that language: the four or five concepts that let you write any tracer on the spot.

01 The shape: probe, filter, action

Every bpftrace program is one or more blocks of the form probe /filter/ { action }. The probe says where to attach (which event). The optional filter (predicate, in / /) is a condition that must be true for the action to run — e.g. only for a certain PID. The action is what to do when the event fires and the filter passes — print something, update a map. That's the entire grammar. A one-liner is just this on the command line with -e; a script is the same blocks in a .bt file. Internalize this shape and the rest is vocabulary.

probe /filter/ { action }probewhere: kprobe:vfs_read/filter/when: /pid==1234/{ action }do: @=hist(arg2)

Fig 1 — The whole language in one line: attach to an event, optionally filter it, then act — usually aggregating into a map.

02 Probe types

The probe names the event source (Chapter 2's probe types, in bpftrace syntax). kprobe:func / kretprobe:func — entry/return of a kernel function. uprobe:/bin/app:func — a user-space function. tracepoint:category:event — a stable kernel tracepoint (preferred). usdt:/path:probe — an app's static probe. profile:hz:99 — timed sampling at 99 Hz (for flame graphs). interval:s:1 — fire every second (for periodic printing). software:/hardware: — perf software events and PMCs. You can use wildcards (kprobe:tcp_*) and list matches with -l.

03 Builtins: reading context

Inside an action, builtin variables give you the event's context for free. The common ones: pid, tid (process/thread ID), comm (process name), uid, nsecs (timestamp in nanoseconds), cpu, arg0..argN (the probed function's arguments), retval (a kretprobe's return value), and kstack/ustack (the kernel/user stack traces). These are the raw material of every tracer — you filter on them, print them, and key your maps by them. Knowing this handful covers the vast majority of one-liners.

04 Variables and maps

Two kinds of storage. Scratch variables ($x) are local to one probe action — handy for a temporary, like saving a start timestamp. Maps (@name) persist across events and aggregate — this is where the in-kernel summarization happens. A bare @ is a single global; @[key] is a hash keyed by anything (e.g. @[comm] per process, @[pid] per process, @start[tid] per thread for timing). The classic latency pattern: save a start time in @start[tid] on entry, compute nsecs - @start[tid] on return, feed it to a histogram. Maps are the heart of bpftrace.

key ideaThe single most useful bpftrace pattern is the per-thread timestamp: store @start[tid] = nsecs on a function's entry probe, then on its return probe compute the delta and aggregate it — @ns = hist(nsecs - @start[tid]); delete(@start[tid]);. That four-line shape measures the latency of any function or operation, custom-filtered however you like. Master it and you can build a latency tool for anything the kernel exposes.

05 Aggregating functions

Maps become summaries through aggregating functions. count() — tally events (@[comm] = count() counts by process). sum(x), avg(x), min(x), max(x) — running totals/stats. hist(x) — a power-of-two histogram (great for latency). lhist(x, min, max, step) — a linear histogram with chosen buckets. These all run in the kernel, so you emit a compact summary, not a flood of events — the overhead win from Chapter 1. bpftrace auto-prints all populated maps when the program ends, so often you just populate a map and quit.

06 Actions and output

Beyond aggregating, actions can print. printf("...", args) formats a line per event (use sparingly — per-event output costs more). print(@map) emits a map on demand. time(), str(ptr) (read a string from a pointer), ksym(addr)/usym(addr) (resolve an address to a symbol), and delete(@map[key]) (free a map entry) round out the common toolkit. The discipline from earlier chapters applies: prefer aggregating into maps over printf-ing every event, so you stay cheap and readable on busy systems.

07 BEGIN, END, and structure

Two special probes bracket a program. BEGIN { ... } runs once at start — print a header, initialize a variable. END { ... } runs once at exit (Ctrl-C) — print final results, clear maps with clear(). A real script combines blocks: a BEGIN banner, one or more event probes that aggregate into maps, maybe an interval:s:1 block to print periodically, and an END to tidy up. This structure scales a one-liner into a readable multi-probe tool while staying far shorter than the equivalent BCC Python+C.

the catchThe seductive trap is reaching for printf on a high-frequency probe — it feels natural ("just print each event") but on a busy system it floods your terminal, drops events when the output buffer overflows, and adds real overhead per event. bpftrace's whole advantage is in-kernel aggregation: a hist() or count() in a map ships one compact summary instead of millions of lines. Reserve printf for low-frequency events or short, filtered captures; for anything hot, aggregate into a map and let bpftrace print the summary at the end.

08 From one-liner to tool: a workflow

(1) Identify the event — list probes with bpftrace -l 'tracepoint:syscalls:*' or wildcards. (2) Start with a count: -e 'tracepoint:... { @[comm] = count(); }' to confirm it fires and see who triggers it. (3) Add a filter to narrow (/comm == "nginx"/). (4) For latency, apply the per-thread timestamp pattern with hist(). (5) If it's useful and reusable, promote it to a .bt script with BEGIN/END — or formalize it as a BCC tool. Build up incrementally: count, filter, time, aggregate. That progression turns a vague question into a precise tracer in minutes.

common catches & gotchas

  • printf on hot probes — Floods output, drops events, adds overhead. Aggregate into maps (hist/count) and print the summary instead.
  • kprobe on a moving target — Dynamic probes break across kernel versions. Prefer tracepoint:/usdt: when available; re-check after upgrades.
  • Forgetting to delete map entries — Per-thread timing maps (@start[tid]) leak if you don't delete() them after use; the map grows unbounded.
  • Unfiltered wide wildcardskprobe:* attaches to a huge number of functions with real overhead. Narrow the wildcard.
  • Broken ustack — User stacks need frame pointers/symbols; otherwise [unknown]. Same frame-pointer issue as Chapter 2.
  • Expecting it to print mid-run — Maps print at END by default. Use an interval:s:1 { print(@); } block for live output.

09 Questions engineers actually ask

What's the basic structure of a bpftrace program?

probe /filter/ { action } — the probe says where to attach, the optional filter is a condition, the action runs when the event fires and the filter passes. One-liners use -e on the command line; scripts put the same blocks in a .bt file. That single shape is the whole language.

How do I measure a function's latency?

Use the per-thread timestamp pattern: on the entry probe save @start[tid] = nsecs; on the return (kretprobe) compute @ = hist(nsecs - @start[tid]) and delete(@start[tid]). That gives a latency histogram for any function, filterable however you like.

What's the difference between $x and @x?

$x is a scratch variable, local to one probe action (e.g. a temporary). @x is a map — it persists across events and aggregates, and bpftrace auto-prints it at the end. Use scratch vars for temporaries, maps for anything you want to summarize or carry between events.

Why is my one-liner flooding the terminal?

You're probably printf-ing on a high-frequency probe. Switch to in-kernel aggregation — count() or hist() into a map — so bpftrace emits one summary instead of a line per event. Reserve printf for low-rate or tightly filtered events.

How do I find what probes are available?

bpftrace -l lists probes, with wildcards: bpftrace -l 'tracepoint:syscalls:*' or 'kprobe:tcp_*'. Prefer tracepoints (stable) over kprobes (version-fragile) when one exists for your question.

10 Key takeaways

  • bpftrace is an awk-like language: probe /filter/ { action } — where, when, what.
  • Probes: kprobe/uprobe (dynamic), tracepoint/usdt (stable), profile/interval (timed). Prefer stable ones.
  • Builtins (pid, comm, nsecs, arg0, retval, kstack) give event context for free.
  • Maps (@name, @[key]) aggregate in-kernel; scratch vars ($x) are local temporaries.
  • The per-thread timestamp pattern + hist() measures any function's latency.
  • Aggregate with count()/sum()/hist() instead of printf-ing hot events — stay cheap.
  • BEGIN/END bracket setup/teardown; build tools up incrementally: count → filter → time → aggregate.
// chapter cheatsheetbpftrace language

structure & probes

probe /filter/ { action }where / when / what.
kprobe: kretprobe: uprobe: tracepoint: usdt:Event sources (prefer tracepoint/usdt).
profile:hz:99 · interval:s:1Timed sampling / periodic firing.
bpftrace -l 'tracepoint:*'List available probes.

builtins (context)

pid tid comm uid cpu nsecsProcess/thread/time context.
arg0..argN · retval · kstack · ustackFunction args/return + stacks.

variables & aggregation

$x = ...Scratch (local) variable.
@[key] = count() / sum(x) / hist(x)In-kernel map aggregation.
lhist(x,min,max,step)Linear histogram with chosen buckets.

the latency pattern

kprobe:f { @s[tid]=nsecs; }Save start time per thread.
kretprobe:f /@s[tid]/ { @=hist(nsecs-@s[tid]); delete(@s[tid]); }Histogram a function's latency.

structure blocks

BEGIN { } · END { clear(@); }Setup / teardown.

11 Wrapping up

bpftrace fits in your head: one grammar, a handful of builtins, maps for aggregation, and the per-thread timestamp pattern for latency. With BCC for finished tools and bpftrace for custom one-liners, you now have the full toolkit — and the rest of the book is applying it, subsystem by subsystem. We start where most analysis begins, with the processor. Next: CPUs.

← prev: Chapter 4next: Chapter 6 →
© cvam — written in plaintext, served warm