PagedAttention — how vLLM stopped wasting 80% of your GPU memory.

Serving an LLM means holding a growing KV cache per request in GPU memory — and until 2023, every serving system allocated that cache as one big contiguous slab sized for the worst case (the maximum sequence length), even though most requests never get that long. The result: 60–80% of expensive GPU memory sat reserved and unused. PagedAttention (vLLM, UC Berkeley, SOSP 2023) borrows the oldest trick in operating systems — virtual memory paging — and applies it to the KV cache: split it into small fixed-size blocks, store them non-contiguously, and track them with a block table, exactly like a CPU's page table. Waste drops to under 4%. Because blocks can now be shared between sequences via copy-on-write, vLLM also gets near-free memory savings for beam search, parallel sampling, and shared prompts. The net effect: 2–4× more throughput on the same GPUs, which is why nearly every serving system shipped since (TGI, TensorRT-LLM, SGLang, LMDeploy) adopted some form of paged or block-based KV cache. This is the whole idea, built from first principles.

The KV cache: the thing that's actually filling your GPU

To understand what PagedAttention fixes, you first need to understand why LLM serving is a memory problem and not just a compute problem. When a transformer generates text autoregressively, it doesn't want to recompute attention over the whole prefix at every step — that would be quadratically wasteful. Instead, it caches the key (K) and value (V) vectors it computed for every previous token, at every layer and every attention head, and reuses them. This is the KV cache, and it is the single largest consumer of GPU memory during inference — often larger than the model's own weights once you have more than a handful of concurrent requests.

The size of the KV cache for one sequence is roughly:

The leading 2 is for storing both K and V. Plug in real numbers for a 13B-parameter model (40 layers, 40 heads, head_dim 128, fp16 → 2 bytes) at a 2048-token sequence: that's roughly 1.7 GB for a single request. Double the sequence length, double the cache. Serve 64 concurrent requests at that length and you need over 100 GB just for KV cache — more than the weights of the model itself. This is why LLM serving is dominated by a memory-management problem: how do you fit as many concurrent sequences' caches into a fixed pool of GPU HBM as possible, without wasting any of it?

The catch is that KV cache is dynamic. Unlike model weights, which are a fixed size known before you start, a request's KV cache grows one token at a time as it generates, and nobody knows in advance exactly how long any given request will run. Before PagedAttention, every serving system had to solve this uncertainty the crude way: pre-allocate a contiguous block of memory sized for the model's maximum context length — say, 4096 or 32768 tokens — for every single request, whether that request ends up generating 12 tokens or 12,000.

Two kinds of waste: internal and external fragmentation

This "reserve the max, hope for the best" strategy causes exactly the same two failure modes operating-systems researchers identified for memory allocation fifty years ago.

Internal fragmentation is memory that's allocated to a request but never used. If you reserve 4096 tokens' worth of KV cache and the request only ever generates 300 tokens before hitting a stop sequence, the other 3796 tokens' worth of memory sat there, fully reserved, fully wasted, for the entire lifetime of that request — unusable by any other request even though it's sitting empty. Across a whole fleet of concurrent requests with a wide, unpredictable spread of output lengths, this waste compounds badly. The original vLLM paper measured that existing systems (at the time: raw HuggingFace Transformers, FasterTransformer, Orca) wasted 60% to 80% of allocated KV cache memory to fragmentation and over-reservation — meaning only a fifth to two-fifths of your expensive GPU HBM was ever doing useful work.

External fragmentation is the second failure mode, familiar to anyone who has watched a long-running process fragment a heap. Even for memory that legitimately gets freed (a request finishes, its cache is released), that freed memory sits as a variable-size hole between other allocations. A new request with a different length requirement may not fit cleanly into any of the available holes, even though the *total* free memory would be more than enough — because contiguous allocation demands a single unbroken run of free bytes, and the holes are scattered and irregularly sized.

The core idea: borrow paging from the operating system

PagedAttention's insight is almost embarrassingly simple once you see it, which is the mark of a genuinely good systems idea: operating systems solved exactly this class of problem decades ago, for exactly the same reason — processes need memory of an unpredictable, growing size, and you don't want to either over-reserve or suffer fragmentation. The OS answer is virtual memory paging: split memory into small fixed-size pages, let a process's logical address space be contiguous while the underlying physical pages are scattered anywhere in RAM, and maintain a page table that translates one to the other. A process never needs to know or care that its "contiguous" memory is actually scattered fragments; the page table hides the indirection.

PagedAttention applies the identical idea to the KV cache. Instead of one giant contiguous tensor per sequence:

• The KV cache is divided into fixed-size KV blocks — vLLM's default is 16 tokens per block (configurable).
• Each block holds the K and V vectors for exactly that many consecutive token positions, across all layers and heads.
• Physical blocks can live anywhere in GPU memory — there is no requirement that a sequence's blocks be contiguous with each other.
• A per-sequence block table (the "page table") maps each logical block index (0, 1, 2, … in the order the sequence uses them) to a physical block address in the global memory pool.
• Blocks are allocated on demand, one at a time, only as a sequence actually generates enough tokens to fill the next one — never all up front for a hypothetical maximum length.

The effect on fragmentation is immediate. Internal fragmentation is capped at a single block. Because blocks are only allocated as they fill up, the worst case waste for any sequence is "the last, partially-filled block" — at most 15 unused token-slots out of a 16-token block, not thousands. Empirically this drops waste from 60–80% down to under 4%. External fragmentation effectively disappears too, because every block is the same fixed size — any free physical block can satisfy any sequence's next allocation request, so there's no such thing as "a hole too small to use." This is precisely why fixed-size paging beats variable-size heap allocation in operating systems, and the argument transfers over unchanged.

The PagedAttention kernel: computing attention over scattered blocks

Splitting storage into blocks is the easy half of the idea. The harder half — and the part that makes this a genuine systems contribution, not just "use a hash map" — is that the attention computation itself now has to read from memory that is no longer contiguous. A standard attention kernel assumes it can point at one flat tensor of shape [seq_len, head_dim] and stream through it. PagedAttention's kernel instead has to, for every query, walk the sequence's block table, and for each logical block, follow the pointer to wherever its physical block actually lives in the memory pool, fetch it, and accumulate the attention score contribution from it — before moving to the next logical block, which might be anywhere else in memory.

Conceptually, the modified inner loop looks like this (simplified from vLLM's actual CUDA kernel, which fuses this into a much more heavily optimized form):

def paged_attention(query, block_table, kv_cache_pool, block_size=16):
    """
    query:          [head_dim]                 the current token's query vector
    block_table:    [num_logical_blocks]       logical -> physical block id
    kv_cache_pool:  [num_physical_blocks, block_size, head_dim]  (K, and separately V)
    """
    scores = []
    for logical_idx, physical_idx in enumerate(block_table):
        # Fetch this block's K vectors wherever they physically live —
        # NOT assumed to be adjacent to the previous block.
        k_block = kv_cache_pool.K[physical_idx]        # [block_size, head_dim]
        scores.append(query @ k_block.T)               # partial attention scores

    scores = softmax(concat(scores))                    # normalize across ALL positions

    output = 0
    for logical_idx, physical_idx in enumerate(block_table):
        v_block = kv_cache_pool.V[physical_idx]         # [block_size, head_dim]
        block_scores = scores[logical_idx * block_size : (logical_idx + 1) * block_size]
        output += block_scores @ v_block                # weighted sum over this block

    return output

Mathematically, this computes exactly the same attention as a standard contiguous kernel — softmax over the same set of keys, weighted sum over the same values. There is no approximation here, unlike, say, sparse or windowed attention. The only difference is the extra indirection: an extra memory fetch through the block table before each block's K/V vectors can be read. This is the same trade operating systems make with virtual memory (a TLB miss costs you a page-table walk) — you pay a small, bounded overhead per access in exchange for eliminating a much larger structural waste. vLLM's actual kernel is heavily optimized (it fuses the block-table lookup into the same kernel that does the QK^T and softmax-weighted-V steps, and later versions integrate FlashAttention-style tiling — more on that below) so in practice the overhead is small relative to the throughput gained from fitting far more concurrent sequences in memory.

Continuous batching: the scheduler that makes blocks worth having

PagedAttention's block-based memory manager is only half of vLLM; the other half is the scheduler, and the two are designed to work together. Older serving systems used static batching: a batch of requests starts together and the whole batch runs until the longest sequence in it finishes, even if every other sequence in the batch finished producing its answer many steps earlier and is just sitting there wasting a GPU slot. vLLM instead uses iteration-level (continuous) scheduling: the scheduler makes a fresh decision every single forward pass — which sequences continue, which finish and free their blocks immediately, and which brand-new requests get admitted into the now-freed capacity — rather than committing to a fixed batch membership for the whole request's lifetime.

This only works efficiently because of PagedAttention. If memory were still allocated as big contiguous reservations, freeing one finished sequence's memory and immediately admitting a new, differently-sized request would run straight back into the fragmentation problem — the freed slab might not be the right shape for the new arrival. Because everything is uniform, fixed-size blocks, any freed block can immediately serve any new request's next allocation, with zero shape-matching problem. Paged memory and iteration-level scheduling are a matched pair: one makes fine-grained admission/eviction decisions possible, the other makes fine-grained memory reclamation possible, and you need both to get the throughput gains.

Memory sharing: copy-on-write for parallel sampling and beam search

Because PagedAttention's block table is just a mapping — like a page table — nothing stops multiple sequences' block tables from pointing at the same physical block. This is the second big win, and it's the one that has no equivalent in the fragmentation story: it lets vLLM share memory across requests that have overlapping content, cheaply and safely.

The clearest case is parallel sampling — a single prompt, multiple independent completions requested at once (common for "generate 4 candidate answers and pick the best"). Every one of those completions shares the exact same prompt, so their KV cache for the prompt tokens is identical. Instead of duplicating that memory once per completion, vLLM lets every completion's block table point at the same physical blocks for the shared prompt prefix, using a reference count per physical block to track how many logical blocks currently point at it.

The catch, and the reason this needs to be more than "just alias the pointer," is that once generation begins, each completion diverges — it appends its own new tokens after the shared prefix. If two sequences shared a block and one of them tried to write a new token into it, it would corrupt the other sequence's cache. vLLM solves this with copy-on-write (COW), again lifted straight from OS memory management: as long as a block is only being read, sharing is free — no copy needed. The moment a sequence needs to write to a shared block (because its own new token lands inside what was previously a shared, read-only block boundary), the memory manager makes a private copy of just that one block for the writer, decrements the original block's reference count, and lets the writer proceed on its private copy. Every other sequence sharing the original block is completely unaffected.

The same mechanism applies to beam search, where multiple candidate beams share a common ancestor path and only diverge at branch points, and to shared system prompts across many different users' requests — a long, identical instruction prefix can be computed once and its blocks shared read-only across every request that uses it, which is functionally an early, block-granularity version of what later systems like SGLang's RadixAttention generalize much further (see below). The original paper reports up to 55% memory savings and as much as 2.2× throughput in parallel-sampling and beam-search workloads specifically, on top of the baseline fragmentation fix.

Preemption: what happens when memory runs out anyway

Paging doesn't create memory out of nothing — it just uses it efficiently. GPU HBM is still finite, and under heavy load vLLM's scheduler will sometimes need to reclaim memory from a running sequence to make room for others (or to avoid deadlock when several sequences are all waiting on more blocks). vLLM supports two preemption strategies, and picks between them based on the situation:

• Swapping — copy the preempted sequence's KV cache blocks out to CPU RAM, and copy them back into freshly allocated GPU blocks when the sequence resumes. This preserves all the computed KV state, so resuming is cheap (no recomputation), but costs PCIe transfer bandwidth and a chunk of host memory.
• Recomputation — simply drop the preempted sequence's KV cache and, when it's resumed, recompute it from scratch by re-running the prefill over the original prompt plus whatever tokens had already been generated. This costs GPU compute time instead of transfer bandwidth, and tends to win when the preempted sequence is short (little to recompute) or when PCIe bandwidth is the tighter resource.

Which strategy is faster depends on the ratio of KV cache size to prefill compute cost for the specific sequence and hardware — vLLM's scheduler picks a policy per situation rather than hardcoding one, and this remains an active area of tuning in production deployments.

The numbers, and why they matter economically

The SOSP 2023 paper's headline result: vLLM delivers 2–4× higher throughput than FasterTransformer and Orca (the strongest serving systems at the time) at the same latency target, across a range of model sizes and workloads — and the gap widens for longer sequences, larger models, and more complex decoding algorithms (beam search, parallel sampling), because those are exactly the cases where the old contiguous-allocation waste was worst. Separately, community benchmarks comparing against naive HuggingFace generate() (no batching optimization at all) have reported gains as high as an order of magnitude or more under favorable conditions — a useful data point, but a much easier baseline to beat than FasterTransformer/Orca, so treat it as directional rather than a controlled comparison.

System / approach	KV cache waste	Batching	Relative throughput
Naive HF generate()	~0% waste, but no cross-request batching at all	none / static per-call	baseline (1×)
FasterTransformer / early Orca	60–80% (contiguous, max-length reservation)	static or coarse-grained	meaningfully faster than naive, still memory-bound
vLLM (PagedAttention)	<4% (capped at last partial block)	iteration-level continuous	2–4× vs. FasterTransformer/Orca

The other side: what PagedAttention costs you

No systems technique is free, and PagedAttention's tradeoffs are exactly the ones you'd expect from adding a layer of indirection. These became sharper once follow-up work went looking for them:

• Non-contiguous kernel overhead. Every attention computation now has to resolve the block table before it can fetch K/V vectors, instead of streaming through one flat tensor. This is extra CPU-side bookkeeping and extra GPU-side indirection on the hot path of every single attention call — small per-call, but it's paid on every call, at every layer, for every token.
• Kernel rewriting and portability. Because the KV cache is no longer a plain contiguous tensor, you can't just plug in any off-the-shelf attention kernel (a new FlashAttention release, a new fused kernel from a hardware vendor) — someone has to rewrite it to understand block tables first. This couples the serving framework tightly to custom kernel code and slows how quickly it can adopt new attention-kernel research.
• Block-size tuning is a real knob. Smaller blocks (e.g. 16) cap internal fragmentation tightly but mean more block-table entries and more indirection per sequence; larger blocks reduce indirection overhead but let more waste creep back in via a bigger "last partial block." vLLM's default of 16 is a reasonable middle ground, not a universal optimum.

This is precisely the critique made by vAttention (Microsoft Research + IISc, arXiv 2405.04437), which argues that PagedAttention solves the fragmentation problem at the cost of software complexity and kernel-portability — and proposes an alternative: use the CUDA driver's actual virtual-memory APIs (which real GPUs and drivers already support) to keep each sequence's KV cache virtually contiguous — so unmodified, off-the-shelf attention kernels keep working unchanged — while still allocating the underlying physical pages on demand, achieving the same "don't reserve the max length up front" benefit without a custom paged kernel at all. It's a nice illustration of how systems research iterates: PagedAttention proved paging works for this problem in software; vAttention asks whether the OS/driver can just do the paging for you, the way it always has for regular process memory.

Where PagedAttention sits in the wider inference-optimization landscape

It's easy to conflate PagedAttention with other "make attention fast" ideas circulating at the same time, so it's worth being precise about the boundaries. FlashAttention (Dao et al., arXiv 2205.14135) is a fast attention compute kernel — it restructures the QK^T-softmax-V computation with IO-aware tiling so it minimizes slow HBM↔SRAM traffic on the GPU. PagedAttention is a memory management scheme for where and how the KV cache is stored between calls. These solve genuinely different problems and are complementary, not competing — later vLLM versions integrate a FlashAttention-based backend that is also paging-aware, so you get both the fast tiled compute of FlashAttention and the fragmentation-free storage of PagedAttention in the same kernel.

System / paper	What it actually optimizes	Relationship to PagedAttention
FlashAttention (2205.14135)	Attention compute — IO-aware tiling to cut HBM traffic	Complementary — different layer of the stack, later fused together
vAttention (2405.04437)	KV cache memory management via CUDA VMM APIs	Alternative — same goal, avoids custom kernel rewrites
SGLang / RadixAttention (2312.07104)	Prefix-sharing across requests via a radix tree, not just block refcounts	Generalization — much richer automatic prefix reuse
TokenAttention (LightLLM)	Token-granularity (not block-granularity) KV cache management	Alternative granularity — finer-grained, different tradeoffs
FlashInfer (2501.01005)	Unified, customizable attention kernel library across paged/sparse layouts	Infrastructure — used by vLLM, SGLang, MLC-Engine to implement paging efficiently

SGLang's RadixAttention (arXiv 2312.07104) deserves a closer look because it takes the sharing idea in PagedAttention and generalizes it substantially. PagedAttention's copy-on-write sharing is opportunistic and mostly limited to cases the application explicitly sets up (parallel sampling from one prompt, beam search). RadixAttention instead maintains a radix tree over all KV cache blocks currently in memory, keyed by token sequence, so that any two requests that happen to share a prefix — even ones the scheduler didn't know in advance were related, like two different users hitting the same long system prompt or few-shot template — automatically discover and reuse each other's cached prefix, with LRU eviction over the tree when memory runs low. It's a more automatic, general-purpose version of the same underlying insight: don't recompute or re-store what's already identical.

TokenAttention, used in the LightLLM serving framework, pushes granularity in the other direction — managing the KV cache at the individual token level rather than in fixed 16-token blocks, which can reduce waste even further at the cost of more bookkeeping entries to track. There's no single dedicated academic paper for it; it's documented primarily in the LightLLM project's own repository and design docs, which is itself a useful reminder that not every influential serving-systems idea gets a standalone arXiv paper — plenty of this field moves through open-source engineering first.

A small worked example: three requests, one memory pool

To make the block-table mechanics concrete, walk through a tiny scenario. Suppose the block size is 4 tokens (smaller than vLLM's real default of 16, just to keep the example short), and the physical memory pool has 6 blocks total, numbered P0–P5, all initially free.

1. Request A arrives, prompt is 5 tokens. That needs 2 logical blocks (4 + 1 tokens). The manager allocates the first two free physical blocks, say P0 and P1. Block table for A: {L0→P0, L1→P1}. Note L1 (physical block P1) only has 1 of its 4 slots used — this is A's entire internal fragmentation, capped at 3 wasted slots.
2. Request B arrives, a parallel-sampling request generating 2 independent completions from the same 4-token prompt. The prompt needs exactly 1 logical block, so both completions' block tables initially point at the same physical block, say P2, with a reference count of 2: B1: {L0→P2}, B2: {L0→P2}, refcount(P2)=2.
3. Both B completions generate their first token. Since P2's single block is already full (4/4), both need a brand-new logical block L1 for their own new token — no COW copy is triggered yet, because they're not writing into the shared block, just adding new blocks after it. Manager allocates P3 for B1's L1 and P4 for B2's L1.
4. Request A finishes (hits a stop token). Its blocks P0 and P1 are immediately freed back to the pool — available for the very next scheduling decision, per the continuous-batching model.
5. Request C arrives needing 2 blocks. Because P0 and P1 are free (and any free block fits any request — no shape-matching problem), the manager allocates them straight to C without any fragmentation concern, even though C's memory requirement has nothing to do with A's.

At every step, memory that isn't actively holding real token data is either a small, bounded partial-block remainder, or immediately-reusable free blocks — never a large reserved-but-empty reservation, and never an awkwardly-shaped unusable hole. That's the entire mechanism, scaled up to real workloads with thousands of concurrent blocks.

A practical tuning playbook

A short history

When	What
Sep 2023	PagedAttention / vLLM paper posted to arXiv (2309.06180); SOSP 2023 publication.
Dec 2023	SGLang / RadixAttention (2312.07104) generalizes KV-cache sharing via a radix tree.
2023–2024	TensorRT-LLM, TGI (Text Generation Inference), and LMDeploy adopt paged or block-based KV cache management of their own.
May 2024	vAttention (2405.04437) proposes CUDA-VMM-based KV cache management as an alternative to custom paged kernels.
Jan 2025	FlashInfer (2501.01005) ships a unified, JIT-customizable kernel library spanning paged and block-sparse KV-cache formats, adopted across vLLM, SGLang, and MLC-Engine.

FAQ

Does PagedAttention change the model's outputs?

No. It is purely a memory-management and storage-layout technique — the attention computation over the same set of keys and values produces mathematically identical results whether they're stored contiguously or in scattered blocks. Output quality and token probabilities are completely unaffected; only memory efficiency and achievable batch size change.

Is PagedAttention the same thing as vLLM?

No — PagedAttention is the specific memory-management technique (the block table + paged KV cache + custom kernel); vLLM is the full serving system that was built around it, which also includes the continuous-batching scheduler, the preemption/swapping logic, and API-serving infrastructure. You'll sometimes see the two terms used loosely interchangeably, but the paper and the system are distinct contributions.

Why 16 tokens per block specifically?

It's an empirically chosen default, not a theoretical optimum — small enough to keep the internal-fragmentation cap (the last partial block) tight, large enough to keep the number of block-table entries per sequence and the per-block bookkeeping overhead manageable. Different workloads with very different sequence-length distributions can benefit from tuning it, but 16 is a reasonable default for general use.

Does this help the prompt-processing (prefill) phase too?

Prefill's memory pattern is different — the whole prompt's KV cache is computed in one parallel pass, so the main win there is compute efficiency (FlashAttention-style kernels), not avoiding gradual fragmentation. PagedAttention's biggest win is in the decode phase and in fitting many concurrent, variable-length sequences into memory at once — though a paged KV cache does also make the prefill-to-decode handoff and prefix sharing across requests possible, which does help prefill indirectly by letting shared prefixes skip recomputation entirely.

How does this relate to quantization?

Orthogonal and composable. Quantizing the KV cache itself (storing K/V in int8 or fp8 instead of fp16) shrinks the per-token memory footprint directly, which is a different lever from PagedAttention's job of eliminating waste in how that memory is allocated. Serving stacks commonly combine both: a quantized, paged KV cache gets you the benefit of smaller per-token cost and near-zero allocation waste at the same time.

What happens if the memory pool genuinely fills up under real load?

The scheduler preempts a running sequence — either swapping its blocks out to CPU RAM (cheap to resume, costs PCIe bandwidth) or dropping them and recomputing via prefill later (costs GPU compute, no transfer cost). Which is chosen depends on the sequence's length and the system's current PCIe-vs-compute bottleneck; it isn't a hardcoded, one-size-fits-all policy.

Do I need to change my model or retrain anything to use PagedAttention?

No — it's entirely a serving-infrastructure change, invisible to the model itself. You point an existing pretrained model's weights at a serving framework that implements PagedAttention (vLLM being the reference implementation), and the memory management happens underneath the model with no retraining or architecture change required.

Is PagedAttention still relevant, or has it been superseded?

It remains the reference technique and is still the mechanism inside vLLM, one of the most widely deployed open-source serving frameworks. Follow-on work (vAttention, RadixAttention, TokenAttention) explores different tradeoffs on the same core problem rather than making paging obsolete — think of it less as "replaced" and more as "the starting point the field is still iterating from."

Why doesn't every serving system just use OS-level virtual memory directly, like vAttention proposes, instead of a custom scheme?

That's precisely vAttention's pitch — and it's a reasonable one. The tradeoff is that CUDA's virtual-memory-management APIs weren't originally designed with this exact workload in mind, so using them well for KV cache management is its own systems-engineering effort with its own rough edges; PagedAttention's custom, software-only approach was simpler to get working first and is battle-tested at scale in vLLM today. Both are valid points on the same design spectrum.

What's the single biggest practical benefit for someone running a serving stack?

More concurrent requests per GPU at the same latency target, which directly lowers the cost per served token — this is the actual economic lever behind "2–4× throughput," and it's why essentially every serious open-source and commercial LLM serving stack now implements some form of paged or block-based KV cache management.

Takeaways

• The KV cache, not the model weights, is usually the dominant and most constrained memory consumer once you're serving more than a handful of concurrent requests.
• Pre-PagedAttention systems reserved KV cache contiguously for the worst-case sequence length, wasting 60–80% of allocated memory to internal and external fragmentation.
• PagedAttention borrows OS virtual-memory paging wholesale: fixed-size KV blocks, non-contiguous physical storage, and a per-sequence block table — dropping waste to under 4%.
• A modified attention kernel walks the block table to gather scattered K/V blocks, computing mathematically identical attention — this is a storage change, not an approximation.
• Continuous (iteration-level) scheduling and paged memory are a matched pair: fine-grained memory reclamation only pays off with fine-grained admission decisions, and vice versa.
• Block tables enable copy-on-write memory sharing across requests with common prefixes — parallel sampling, beam search, shared system prompts — for up to 55% additional memory savings.
• The net result reported in the original paper: 2–4× throughput over FasterTransformer/Orca at equal latency, which is why nearly every serving framework since has adopted some form of paged or block-based KV cache.
• It isn't free: extra kernel indirection, coupling to custom attention kernels, and a block-size tuning knob — tradeoffs that vAttention specifically argues can be avoided using native CUDA virtual-memory support instead.
• PagedAttention (memory layout) is complementary to, not competing with, FlashAttention (compute kernel) — modern stacks use both together.

References & further reading

• Kwon et al. — Efficient Memory Management for Large Language Model Serving with PagedAttention (SOSP 2023) — the original paper; block tables, the paged kernel, and the 2–4× throughput results.
• vLLM documentation — PagedAttention design — the reference implementation's own explanation of the kernel and block manager.
• Prabhu et al. — vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention — the CUDA-VMM alternative, and the clearest published critique of PagedAttention's tradeoffs.
• Zheng et al. — SGLang: Efficient Execution of Structured Language Model Programs — RadixAttention, generalizing PagedAttention-style sharing via a radix tree over prefixes.
• Dao et al. — FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — the complementary compute-kernel optimization, later fused with paged storage.
• Ye et al. — FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving — unified kernel library spanning paged/block-sparse KV-cache formats, used by vLLM and SGLang.
• Zhou et al. — A Survey on Efficient Inference for Large Language Models — survey paper; positions PagedAttention within data/model/system-level inference optimizations.
• Miao et al. — Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems — survey paper covering serving-system design broadly, including KV cache management.
• Taming the Titans: A Survey of Efficient LLM Inference Serving — survey paper; instance-level and cluster-level serving strategies.
• LLM Inference Serving: Survey of Recent Advances and Opportunities — survey paper covering recent serving-system advances.