Rook: storage for Kubernetes, from the maintainers.

Deep dive 16 is the maintainer talk for Rook — the CNCF graduated operator that runs Ceph storage inside Kubernetes. The maintainers (Deepika Upadhyay of Clyso; Madhu Rajanna, Rewant Soni, Malay Parida, Pratik Surve of IBM) walked the full stack: what Rook is, the three-layer architecture (Rook operator → Ceph-CSI → Ceph data plane), the new features landing in Rook v1.20 (NVMe-oF gateway, RGW accounts, the standalone Ceph-CSI operator, two-node clusters), erasure coding as a cheaper alternative to replica-3 — with the big "Fast EC" performance jump in Ceph v20 Tentacle — and end-to-end application disaster recovery via Ramen, OCM, and RBD mirroring. Rook turned 10 this year (created November 2016).

This is the storage anchor of the observability/edge/storage block. Where DD15 (KubeEdge) pushed compute to the edge, this one answers the harder question every stateful workload eventually asks: where does the data live, how does it survive a node — or a whole region — dying, and how do I get all of that without becoming a Ceph operator myself?

What Rook is — and what it is not

Rook is a Kubernetes operator that brings Ceph storage into your cluster. It is not a storage system of its own — Ceph is the storage system. Rook is the automation layer that deploys Ceph, configures it, upgrades it, and exposes its capabilities to your applications as ordinary Kubernetes objects: StorageClasses and PersistentVolumeClaims. You ask for a PVC; under the hood Rook and Ceph satisfy it.

The pitch, distilled from the maintainers' own bullets:

• Brings Ceph storage into your Kubernetes cluster.
• Manages that storage with an operator and CRDs — declarative, GitOps-friendly.
• Automates deployment, configuration, and upgrades — the parts that historically made Ceph intimidating.
• Lets apps consume storage like any other Kubernetes storage — StorageClasses, PVCs, nothing exotic.
• Open source, Apache 2.0.
• Turned 10 in 2026 — created November 2016, now CNCF graduated.

Ceph in one slide — block, file, object

Ceph is an open-source, distributed enterprise storage platform, and its big trick is that a single cluster serves all three storage shapes an application could want:

Shape	Access mode	Ceph component	Typical consumer
Block	RWO (ReadWriteOnce)	Ceph RBD (RADOS Block Device)	databases, single-writer volumes, VM disks
File	RWX (ReadWriteMany)	CephFS	shared multi-node file access
Object	S3 API	Ceph RGW (RADOS Gateway)	buckets, backups, media, data lakes

That "all-in-one" property is the reason Ceph is such a natural fit for Kubernetes: one storage backend covers the RWO database PVC, the RWX shared-content mount, and the S3 bucket your app writes to — instead of bolting together three separate systems.

The three architectural layers

The maintainers were careful to separate the stack into three responsibilities, because confusing them is the source of most operational mistakes:

• Rook — the operator. It deploys and manages Ceph. It does not sit in the data path.
• CSI (Ceph-CSI) — the driver that dynamically provisions and mounts storage into your application pods. This is the layer that turns a PVC into a mounted device.
• Ceph — the data layer. MONs (cluster map / quorum), MGR (management + metrics), OSDs (one per disk, the actual data), MDS (CephFS metadata), RGW (S3 gateway).

Where Rook runs — anywhere Kubernetes runs

The maintainers made a point of the deployment flexibility, because storage choices usually dictate where you can run:

• Cloud — back OSDs with EBS, Persistent Disks, or Azure disks for elasticity. You trade some control for managed block devices.
• On-premises / bare metal — use local SSDs/HDDs directly for performance and control. This is where Ceph historically shines.
• Hybrid / multi-cloud — mix and match for resilience, presenting a unified storage layer across providers.

The slogan on the slide was simply: "Anywhere Kubernetes runs." The substantive claim underneath it is that Rook gives you one storage abstraction whether your disks are cloud volumes or rack-mounted spinning rust.

Why choose Rook — the problems it solves

Challenge	How Rook / Ceph helps
Fear of data loss — no storage spread across AZs	Replicates and distributes data across availability zones via CRUSH failure domains
Limited PVs per node, poor performance of small PVs	Effectively unlimited scaling with dynamic, optimized data placement
Lack of cross-cloud support	A unified storage layer across multiple cloud providers

And on scale specifically, two properties matter:

• Add capacity without downtime — Ceph is built to scale up and out; you add OSDs and the cluster rebalances live.
• Thin provisioning = efficient use — adding storage capacity is independent of the PVCs that consume it, so you provision generously and only pay for what's actually written.

Recent features — what's new in Rook

A large chunk of the talk was the maintainers' tour of what landed recently. These are worth knowing because they change what Rook can do for you in production.

The standalone Ceph-CSI operator

Historically, Ceph-CSI was configured indirectly through Rook operator settings. That coupled the CSI driver's lifecycle to the storage backend orchestrator. The new Ceph CSI Operator is a native Kubernetes operator built specifically to manage the Ceph-CSI plugins:

• Enabled by default with Rook in v1.18+.
• In Rook v1.20, all Ceph-CSI configuration is done via CRs instead of Rook operator settings — fully declarative.
• It decouples the client-side CSI driver lifecycle from the core storage backend, so you can upgrade the driver independently.

It introduces a clean set of CRDs: OperatorConfig (operator-level settings and driver overrides), Driver (installation and lifecycle of a CSI driver), CephConnection (how to reach a Ceph cluster), ClientProfile (CephFS / RBD / NFS / NVMe-oF config used for communication), and ClientProfileMapping (maps client profiles between two clusters — the backbone of backup and DR). The operator configures the RBD, CephFS, NFS, and the new NVMe-oF drivers from those CRs.

RGW accounts

A new CephObjectStoreAccount CRD configures RGW accounts for accessing the S3 API — bringing S3 account management into the declarative Kubernetes model. Experimental in Rook v1.20.

NVMe-oF gateway

NVMe over Fabrics lets Ceph RBD (block) volumes be accessed via the NVMe/TCP protocol. The point: external clients outside the Kubernetes cluster can connect to Ceph block storage using standard NVMe-oF initiators, getting low-latency, high-throughput block access over existing network infrastructure. Experimental — the maintainers explicitly asked for feedback.

Two-node clusters

For minimal edge deployments where a third node is a luxury, Rook now supports two-node clusters: it runs three mons where one of them floats between the two nodes for resiliency, and it requires a fencing solution with Pacemaker. (The maintainers pointed to a dedicated follow-up talk — "When the Edge Can't Afford a Third Node: A Storage Solution for Two-Node Kubernetes Clusters.") This connects directly to the edge themes from DD15.

The Ceph-CSI driver up close

The Ceph Container Storage Interface project hosts the CSI drivers for Ceph. There are four:

Driver	Backs	Notes
RBD	RADOS Block Device	block storage, RWO; RWX block mode for KubeVirt VMs (supports live migration)
CephFS	Ceph Filesystem	RWX filesystem mode, simultaneous multi-node file access
NFS	Network Filesystem	CephFS volumes exposed over NFS for external systems / limited kernel drivers
NVMe-oF	RBD over NVMe-oF	new; RBD volumes exposed over NVMe for external clients

Ceph-CSI internals

A few internals worth carrying around:

• Single container image supports all the different drivers.
• RADOS OMAP for state & checkpointing — Ceph-CSI stores volume and snapshot state information in RADOS OMAP (object key-value maps). This matters later: erasure-coded pools don't support OMAP, which is why metadata needs a separate replicated pool.
• Go-Ceph — Go bindings for the native Ceph C APIs, so the driver can call Ceph directly rather than shelling out.

Standard and recent CSI features

Standard Kubernetes CSI features Ceph-CSI supports: PVC expansion; create/restore from snapshots and clone volumes; topology awareness (read from the OSD nearest the client); ephemeral volumes. Non-Kubernetes-native extras: PVC encryption — LUKS for RBD, fscrypt for ext4/CephFS — with KMS backends including Vault, Azure, IBM HPCS, and KMIP.

Recently added:

• VolumeGroup Snapshots — label PVCs to create consistency groups.
• RBD Block Usage Metrics — image size and number of blocks allocated.
• RBD Change Block Tracking (CBT) — efficient differential backups by comparing snapshots.
• Auto Fencing Support — prevents data corruption by blocking misbehaving clients.

CSI-Addons

CSI-Addons hosts extensions to the CSI specification that provide advanced storage operations. The components are a CSI-Addons Controller (takes a CR from the user), a CSI-Addons Sidecar next to the CSI driver pod, and the CSI driver itself — wired together over gRPC. The operations it unlocks in Ceph-CSI:

• Reclaim Space — runs rbd sparsify on images and fstrim on filesystem-mode volumes to reclaim unused space.
• Network Fence / Network Fence Class — advertise the node's client IP, and an API to block given CIDR IP ranges. Critical in Metro DR and node-loss scenarios.
• Encryption Key Rotation — rotate Key Encryption Keys (KEKs) for encrypted volumes.
• Volume / VolumeGroup Replication — common, reusable APIs for storage DR; enable/disable mirroring and change primary/secondary state of RBD mirrored groups/images. Automates rbd-mirroring with promote, demote, resync, and status. Critical for regional DR — and the foundation for the application DR section below.

Erasure coding — durability without the 3× tax

This was the most quantitative part of the talk, and it's where real money is saved. The maintainers framed it bluntly: "Why make 3 copies when maths can do the job?"

Most Ceph clusters default to replica-3 for durability — three full copies, which means a 3× storage requirement. Erasure coding (EC) provides similar durability with far less overhead, the same way RAID parity does: split data into k data chunks, compute m parity chunks, and you can lose any m chunks and still rebuild.

The headline number: EC 4+2 gives the same durability as replica-3 for roughly 50% less storage. For an EC profile k+m: you need a minimum of k+m failure domains, and you tolerate m simultaneous failures.

How Rook simplifies EC setup

Setting up EC in raw Ceph means defining an EC profile (k, m, plugin like isa, technique like reed_sol_van), a CRUSH rule for chunk placement, then a pool using that profile. Rook collapses all of that: you define a Kubernetes CRD (CephBlockPool / CephFilesystem / CephObjectStore) with an erasureCoded block, and the Rook operator creates the EC profile, CRUSH rule, and erasure-coded pool for you — consumed by RBD, CephFS, or RGW.

# Replicated metadata pool
apiVersion: ceph.rook.io/v1
kind: CephBlockPool
metadata:
  name: replicated-metadata-pool
  namespace: rook-ceph
spec:
  failureDomain: host
  replicated:
    size: 3
---
# Erasure-coded data pool
apiVersion: ceph.rook.io/v1
kind: CephBlockPool
metadata:
  name: ec-data-pool
  namespace: rook-ceph
spec:
  failureDomain: host
  erasureCoded:
    dataChunks: 4
    codingChunks: 2
  parameters:
    allow_ec_optimizations: "true"

When NOT to use EC

The maintainers were honest about the trade-offs:

• Performance cost — higher CPU usage from chunking and parity computation; slower writes than replication; not ideal for small or latency-sensitive workloads.
• Choose k+m carefully — it fixes both capacity efficiency and failure tolerance, requires k+m failure domains, and cannot be changed later. Use m ≥ 2 in production.
• Workload fit — ideal for archives, backups, media, ML datasets, cold storage. Not ideal for databases, VM disks, transactional systems.

"Fast EC" — Ceph v20 Tentacle

The traditional "EC is slow for small I/O" caveat is being dismantled. Ceph v20 (Tentacle) introduced major EC performance improvements:

• Partial Reads — read only the required shards instead of reconstructing the full stripe, slashing small-read latency.
• Partial Writes — update only affected chunks instead of rewriting the whole stripe, saving drive operations.
• Parity Delta Writes — update parity using delta calculations rather than full recomputation, cutting CPU and network overhead.
• Larger Striping — the default 16 KB stripe_unit improves block and file performance significantly.

The maintainers' performance charts (1024K random read and write) showed that with large reads and writes, EC performance is almost identical to replica-3 — and "fast EC" (tentacle-fast-ec-6+2-16k) closes much of the remaining gap. Coming in Ceph v21 (Umbrella), late 2026: Direct Reads (clients talk to OSDs directly, matching replicated read performance), OMap support on EC pools (no more separate metadata pool), and Pool Migration (convert between replication and EC, or change k/m, with internal data migration).

Application disaster recovery

The final act tied storage replication to whole-application survival. The framing question: Kubernetes keeps your apps running through pod and node failures and self-heals within a cluster — but what does it not protect against? Entire cluster failure, data corruption, human error, storage backend failure. DR covers exactly that gap: availability across clusters.

RPO and RTO — the two numbers

Objective	Question it answers	Means
RPO — Recovery Point Objective	"How far back can you afford to go?"	How much data you can lose — the max age of data that must be recovered (e.g. "at most 5 mins")
RTO — Recovery Time Objective	"How long can you afford to be down?"	How fast you recover — the targeted restore duration (e.g. "back online within 15 mins")

Your whole DR strategy is designed around these two numbers — and the lower they are, the more it costs.

The three components: OCM, Ramen, Rook

• OCM (Open Cluster Management) — the centralized control plane across clusters. Manages cluster and application lifecycle, handles application placement across regions, and integrates with Ramen for DR orchestration.
• Ramen — the open-source DR controller built on Kubernetes APIs. A hub orchestrator with managed-cluster controllers; it manages DR policies, replication, and failover, using Ceph RBD mirroring and VolSync for the actual data movement.
• Rook (VR & VGR) — deploys and manages the Ceph storage, enables RBD mirroring between clusters, and exposes VolumeReplication (per-PVC replication policy and lifecycle) and VolumeGroupReplication (group PVCs into one consistency unit).

How Rook enables replication for DR

Two mechanisms:

• RBD Mirroring (CephRBDMirror CRD) — creates the RBD pool and enables mirroring, bootstraps peers between clusters, deploys the rbd-mirror daemon for async replication, uses snapshot-based sync (efficient and crash-consistent), and runs in both directions.
• Volume Replication CRDs (CSI-Addons, VR & VGR) — VolumeReplicationClass (driver config + parameters), VolumeReplication (replicates a single PVC), VolumeGroupReplication (replicates multiple PVCs as a consistency group). These manage failover/failback via promote/demote and integrate with Ramen and OCM.

The DR workflow, end to end

1. Deploy — app deployed on the primary cluster (Region 1).
2. Protect — enable DR protection from the Hub.
3. Replicate — data continuously replicates to Region 2.
4. Failover — disaster hits; the Hub initiates failover.
5. Failback — relocate back to Region 1 once recovered — ideally RPO = zero, no data loss.

Appendix — what actually runs (Rook pods)

For grounding, a Rook/Ceph cluster's pods across three nodes typically include: rook-operator and rook-discovery; the CSI pods (csi-provisioner, csi-rbd-plugin, csi-cephfs-plugin); and the Ceph daemons — ceph-mon (quorum, usually 3), ceph-mgr, many ceph-osd (one per disk), plus ceph-mds (CephFS), ceph-rgw (S3), ceph-nfs, ceph-rbd-mirror (DR), and ceph-crash-collector. The data path stays app → CSI kernel driver → OSDs, with each client connected to more than one OSD/RGW.

FAQ

Is Rook the storage, or is Ceph the storage?

Ceph is the storage system. Rook is the Kubernetes operator that deploys, configures, and manages Ceph and exposes it through StorageClasses and PVCs. Rook is control-plane only — it isn't in the I/O path, so it can restart without interrupting running volumes.

When should I pick erasure coding over replica-3?

When you're storing large, throughput-oriented, less latency-sensitive data — archives, backups, media, ML datasets, cold storage — and want to cut raw capacity roughly in half (EC 4+2 ≈ replica-3 durability at ~50% the storage). Avoid EC for databases, VM disks, and transactional systems. Remember k+m can't be changed later, and use m ≥ 2.

Why does an erasure-coded pool need a second pool?

EC pools don't support RADOS OMAP, and RBD/CephFS/Object all store metadata in OMAP. So you pair the EC data pool with a small replicated metadata pool. Ceph v21 (Umbrella) adds OMAP support on EC pools, which will remove this requirement.

How does Rook handle cross-region disaster recovery?

Through three layers: OCM manages clusters and app placement, Ramen is the DR controller orchestrating policies and failover, and Rook enables Ceph RBD mirroring plus VolumeReplication / VolumeGroupReplication CRDs to asynchronously replicate PVCs between regions. The workflow is deploy → protect → replicate → failover → failback.

Is "EC is slow" still true?

Less and less. Ceph v20 (Tentacle) added partial reads, partial writes, parity delta writes, and larger default striping — "Fast EC." The maintainers' benchmarks show large read/write EC performance almost identical to replica-3. Ceph v21 adds direct reads to match replicated read latency further.

Takeaways

• Rook = "Ceph without a Ceph team." An operator that deploys, configures, upgrades, and exposes Ceph as ordinary PVCs and StorageClasses — control-plane only, never in the data path.
• One backend, three shapes — block (RBD, RWO), file (CephFS, RWX), object (RGW, S3) from a single Ceph cluster.
• The Ceph-CSI operator is the new config surface — in Rook v1.20 all CSI config is declarative CRs, decoupled from the storage backend lifecycle.
• Erasure coding saves ~50% raw capacity at replica-3 durability, with the OMAP/metadata-pool caveat — and "Fast EC" in Ceph v20 erases most of the old performance penalty.
• Application DR is real and built-in — OCM + Ramen + Rook RBD mirroring / VolumeReplication give cross-region failover and failback for both declarative and discovered apps.
• 10 years in, CNCF graduated — Rook is the default answer for stateful storage on Kubernetes when you control your own infrastructure.

Next in the series — Deep dive 17: the bonus session that closes out Day 1.

References

• KubeCon Mumbai 2026 — Day 1 index · the rest of the series
• Rook — website & docs · the operator, CRDs, and guides
• Ceph — erasure coding · profiles, k+m, plugins
• Ceph-CSI · the CSI drivers and CSI-Addons
• Ramen — DR controller · OCM-integrated application DR