← Systems Performance

BOOK NOTES · SYSTEMS PERFORMANCE · CHAPTER 1

Systems Performance Chapter 1 — Introduction.

systems-performancechapter-1linuxlatencyobservabilityperformance

// the one-minute version

Systems performance is the study of an entire computer — all software and all hardware, from the application down to the bare metal — working together. It is hard because a request crosses dozens of layers, and the real bottleneck is rarely where you first look. The single most useful metric is latency (how long something took), because it ties directly to user pain and is easy to reason about. Two ways to learn what a system is doing: observability (watch it) and experimentation (poke it and measure). The rest of the book is methods and tools for doing both without guessing.

A user says "the site feels slow." You open a dashboard. CPU is at 40%, memory looks fine, the disk is quiet. Everything is "green" and yet the complaint is real. This is the everyday reality of systems performance: the problem is somewhere in a stack of a dozen layers, the obvious metrics lie by omission, and your job is to find the truth fast. Chapter 1 sets up the whole game — what you are actually analyzing, why it resists easy answers, and the mindset that the rest of the book turns into concrete tools.

01 What "systems performance" actually means

Systems performance is the study of a whole system: the application, the libraries it calls, the system calls into the kernel, the kernel's schedulers and memory manager, the device drivers, and the physical hardware — CPUs, memory, disks, network cards. Most engineers specialize in one slice. Systems performance is the discipline of seeing all the slices at once, because a slow request almost never respects your team's org chart.

The goal is usually one of three things: make something faster (lower latency), let it handle more (higher throughput), or do the same work on less hardware (lower cost). These often trade against each other, and which one matters depends entirely on the business. A trading system cares about the worst-case microsecond; a batch pipeline cares only about total throughput; a startup cares about the cloud bill.

think of it likeDiagnosing a city's traffic jam. You can't fix it by staring at one intersection. The jam might be caused by a closed road three miles away, a broken signal, or simply rush-hour volume. Systems performance is traffic engineering for a computer: you follow the flow, find where it actually backs up, and resist blaming the first red light you see.

02 Why it is genuinely hard

Performance work has a reputation for being mysterious, and there are real reasons for that.

The stack is deep

One web request touches the app, runtime, libraries, syscalls, kernel, drivers, and hardware. A bottleneck in any layer slows the whole thing, and each layer has its own tools and vocabulary.

It's a system, not a part

Components interact. Fixing the disk can expose a lock. Adding CPUs can make a single-threaded app no faster. Effects are non-local and sometimes counterintuitive.

Bottlenecks move

Solve one and the next appears somewhere else. Performance is a moving target — there is always a limiting resource, only the question of which one.

It can be subjective

"Slow" has no units until you measure. Without latency numbers and a target, you can't tell when you're done, or whether a change helped at all.

key ideaPerformance is a property of the whole system under a specific workload, not a fixed number stamped on a component. The same server is "fast" or "slow" depending on what you ask it to do. That is why methods matter more than memorized facts: the answer changes, but the way you find it does not.

03 The full stack you are analyzing

Keep this mental picture. A request enters at the top and travels down through layers until it hits hardware, then results travel back up. Any layer can be the slow one.

One request crosses the entire stackAPPLICATION  ·  your code, runtime, librariesSYSTEM LIBRARIES & SYSCALLSKERNELscheduler · virtual memory · file systems · TCP/IP · driversDEVICE DRIVERSHARDWARECPUs · memory · disks · network interfacesrequest flows downresult flows up

Fig 1 — The system stack. Latency added at any layer is latency the user feels. Your job is to find which layer.

This picture is why the book is organized by resource — CPUs, memory, file systems, disks, network — with a chapter of methods first. You learn one repeatable way to interrogate each layer, then apply it wherever the trail leads.

04 Latency: the metric that matters most

If you remember one word from this book, make it latency — the time for an operation to complete. A disk read latency of 2 ms, a request latency of 100 ms, a lock-wait latency of 5 ms. Latency is powerful because it directly maps to suffering (users wait) and because it composes: the request latency is the sum of the latencies of its parts, so you can break a slow request down and find the dominant cost.

Contrast latency with metrics like IOPS, throughput, or utilization. Those are useful, but they are secondary — you can have high IOPS and a happy system, or low IOPS and a miserable one. Latency tells you whether there is a problem; the other metrics help explain why. Gregg's advice: when you can, measure and target latency first.

watch outAverages hide pain. An average latency of 20 ms can include a tail where 1% of requests take 2 seconds — and that 1% is often your most important users or your retry storms. Always look at the distribution: percentiles (p99, p99.9), maximums, and histograms. The average is the most comforting and most misleading number on the dashboard.

05 Two ways to know: observability and experimentation

There are exactly two families of activity in performance work.

Observability

Watch the system as it runs, without changing it. Counters, tracing, profiling, logs. This is where you spend most of your time, and the bulk of this book. The golden rule: observation should be safe and low-overhead in production.

Experimentation

Apply a deliberate, controlled load and measure the response — benchmarking, load tests, micro-benchmarks. Powerful for "what if," but easy to get wrong and to mislead yourself (Chapter 12 is a list of those traps).

Most real investigations are observability-led: you watch production, form a hypothesis about the bottleneck, and only then run an experiment to confirm a fix. Leading with benchmarks — "let's see how fast it can go" — tends to produce impressive numbers that have nothing to do with the actual problem.

06 Counters, tracing, and profiling

Observability tools come in three flavors, and knowing which is which saves hours.

Counters are numbers the kernel and apps keep all the time — bytes sent, context switches, page faults. Cheap to read; tools like vmstat and iostat just print them. Tracing records individual events as they happen — every syscall, every disk I/O — giving rich detail at higher cost; strace, ftrace, and BPF live here. Profiling takes samples at intervals (say, the running function 99 times a second) to build a statistical picture of where time goes; this is what produces flame graphs.

key ideaCounters tell you something is off. Profiling tells you where the time concentrates. Tracing tells you exactly what happened for a specific event. Start cheap and broad (counters), then go deep and specific (tracing) only on the suspect. That progression — broad to narrow — is the spine of every method in the book.

07 Utilization, saturation, errors

Chapter 2 formalizes a method around three quantities you will meet for every resource, so meet them now. Utilization is how busy a resource is (a disk 90% busy). Saturation is the work that couldn't be serviced and had to queue (the run-queue length, the I/O wait queue). Errors are failed operations (dropped packets, failed mallocs). A resource that is saturated is actively hurting you, even if utilization isn't quite 100% — queueing is where latency is born.

the catchHigh utilization is not automatically a problem, and low utilization does not mean a resource is innocent. A disk at 100% utilization may be fine if nothing is waiting; a CPU at 50% can still be the bottleneck if a single thread is pinned at 100% of one core while the others idle. Per-resource, per-thread detail beats the aggregate average every time — the headline percentage is where investigations go to die.

08 Cloud computing changes the rules

This edition is "Enterprise and the Cloud" for a reason. In the cloud you share hardware with strangers (multi-tenancy), so a noisy neighbor can steal your CPU cycles or disk bandwidth and never show up in your own metrics. You also hit hard limits — the provider caps your IOPS or network, and you saturate a ceiling you can't see in top. And instances are disposable, which changes the economics: sometimes the fix is "add another node," sometimes that just multiplies a per-node inefficiency. Cloud performance gets its own chapter (11), but keep the multi-tenant mindset from the start.

common catches & gotchas

  • Blaming the first green-ish metric — CPU at 40% does not exonerate the CPU; a single saturated core or a hot thread can bottleneck while the average looks calm. Always check per-CPU and per-thread.
  • Trusting averages — The mean smears the tail you care about. Demand percentiles and histograms before you conclude anything about latency.
  • Benchmarking first — Running a load test before you understand the workload usually measures the benchmark's flaws, not your system. Observe production first.
  • Confusing utilization with saturation — 100% busy is not the alarm; queueing (saturation) is. A resource can be 100% utilized and perfectly healthy if nothing waits.
  • Observer effect — Heavy tracing can slow the very thing you measure. Know each tool's overhead before pointing it at production.
  • Stopping at the symptom — "CPU is high" is a symptom, not a cause. Keep drilling until you reach the function, query, or config that actually drives it.

09 Questions engineers actually ask

Where do I even start when something is "slow"?

Get a latency number and a target first ("p99 is 800 ms, we want 200"). Then run a quick broad checklist (Chapter 2's USE method and the "60-second" tool tour) to spot the obviously stressed resource, and drill down from there. Never start by guessing a cause.

Is high CPU usage bad?

Not by itself. High utilization with low latency and no saturation is a system doing its job efficiently. CPU is "bad" only when it causes queueing or steals time from work that matters. Look at run-queue length and per-thread breakdown, not just the percentage.

Why not just add more hardware?

Sometimes that's the right call. But if the app is single-threaded, or the bottleneck is a lock or a slow query, more hardware does nothing — or multiplies the waste. Measure first so you scale the resource that's actually limiting you.

What's the difference between tracing and profiling?

Tracing records every event of a type (every syscall) — complete but expensive. Profiling samples at intervals to estimate where time is spent — cheap and great for "where is the CPU going," which is exactly what flame graphs show.

Do I need to know kernel internals?

Enough to read the metrics, yes — which is why Chapter 3 exists. You don't need to write a scheduler, but knowing what a context switch, page fault, or run queue is turns cryptic counters into a clear story.

10 Key takeaways

  • Systems performance studies the whole machine — app to hardware — because bottlenecks ignore team boundaries.
  • It's hard because the stack is deep, components interact, and the limiting resource keeps moving.
  • Latency is the most useful metric: it maps to user pain and breaks down cleanly into parts.
  • Two activities only: observability (watch) and experimentation (poke). Observe first, benchmark second.
  • Tools are counters (cheap, broad), profiling (where time goes), and tracing (exactly what happened) — use them broad-to-narrow.
  • For every resource, watch utilization, saturation, and errors — and never trust the average over the distribution.
  • The cloud adds multi-tenancy, hidden limits, and noisy neighbors to every problem.
// chapter cheatsheetfirst-look toolbox

the 60-second triage (run these in order)

uptimeLoad averages (1/5/15 min). Rising trend = growing demand. Compare to CPU count.
dmesg | tailRecent kernel messages — OOM kills, disk errors, TCP drops. Often the smoking gun.
vmstat 1System-wide CPU, memory, swap, I/O per second. Watch r (run queue) and si/so (swapping).
mpstat -P ALL 1Per-CPU breakdown. Catches one hot core hiding behind a calm average.
pidstat 1Per-process CPU over time, rolling. Which process is actually burning the CPU.
iostat -xz 1Per-disk utilization, latency (await), queue. The disk truth.
free -mMemory used vs available vs cache. "used" minus cache is what matters.
sar -n DEV 1Network interface throughput per second. Near link/limit ceiling?
top / htopLive overview of top consumers. Good for a glance, weak for trends.

the three things to ask every resource

utilizationHow busy is it? (e.g. %util in iostat, %CPU in top)
saturationIs work queueing? (run-queue r, await, backlog) — this is where latency is born.
errorsAny failures? (dropped packets, I/O errors, failed allocs in dmesg)

mindset

latency firstGet a number + target before touching anything.
broad → narrowCounters to find the suspect, then tracing/profiling to convict it.
distribution > averageAlways check p99 and histograms, never just the mean.

11 Wrapping up

Chapter 1 is the orientation map. The system is the whole stack; the metric is latency; the work is observe-then-experiment; the tools are counters, profiling, and tracing used broad-to-narrow; and every resource gets the same three questions — utilization, saturation, errors. Nothing here is a recipe yet. The recipes start next, with the methodologies that turn this mindset into a repeatable checklist you can run under pressure.

← chapter indexnext: Chapter 2 →
© cvam — written in plaintext, served warm