pgBackRest with S3 — the complete hands-on guide to backup & restore.

pgBackRest is the de-facto standard for serious PostgreSQL backups: parallel, incremental, checksummed, encrypted, and S3-native. But between the docs and a working production setup lies a minefield of hiccups — archive_command silently failing, stanza mismatches after a restore, S3 TLS errors, incremental backups that mysteriously become fulls, WAL piling up until the disk fills, and restores that fail at 2 AM because nobody ever tested them. This guide walks the whole path hands-on: install, S3 repository config, stanza creation, full/differential/incremental backups, verification, PITR, delta restore — then a long troubleshooting section of the exact errors you'll hit and what they actually mean, the hidden configs almost nobody sets (block incremental, file bundling, async archiving, expire-auto), and the production practices that separate "we have backups" from "we can restore."

Why pgBackRest (and why not pg_dump)

First, clear the biggest conceptual confusion in PostgreSQL backups: pg_dump is not a backup strategy. It's a logical export — a snapshot of your data as SQL statements or a custom archive. It's great for migrating a schema, copying one database, or seeding a dev environment. It is terrible as your production disaster-recovery plan, for three reasons: it can't do point-in-time recovery (you restore to exactly when the dump ran, losing everything after), it takes and restores slowly at scale (restoring a multi-TB dump means replaying every INSERT and rebuilding every index), and it puts a real load on the source while it runs.

Real PostgreSQL backup is physical backup + WAL archiving: copy the actual data directory files (fast, block-level), and continuously archive the write-ahead log (WAL) segments that record every change. With a base backup plus a continuous stream of WAL, you can restore to any point in time between the backup and now — down to the second, or to a specific transaction. This is what pgBackRest orchestrates, and it adds everything the raw approach lacks:

• Parallelism — backup and restore with N processes (process-max), saturating your network/disk instead of single-threading like a bare pg_basebackup.
• Incremental & differential backups — copy only changed files (or, with block incremental, only changed parts of files) instead of the whole cluster every time.
• Native object storage — S3, Azure Blob, and GCS support built in, no fuse mounts or gateway hacks.
• Checksums everywhere — every file checksummed at backup time and verified at restore; page-level checksum validation of the data itself during backup.
• Encryption at rest — AES-256-CBC of the repository, independent of whatever the storage provides.
• Retention management — automatic expiry of old backups and their WAL by count or time, so the repo doesn't grow forever.
• Delta restore — restore only the files that differ from what's already on disk, turning a multi-hour restore into minutes when most data is intact.

Tool	Type	PITR	Parallel	Incremental	S3 native	Best for
pg_dump	logical	no	partially (-j for directory format)	no	no	migrations, dev seeds, single-table recovery
pg_basebackup	physical	with manual WAL setup	no	PG17+ only, limited	no	quick replica seeding, tiny clusters
pgBackRest	physical	yes, first-class	yes	full/diff/incr + block-level	yes	production DR, anything that matters
Barman	physical	yes	yes (rsync/ssh modes)	yes	via barman-cloud	similar niche, different tradeoffs
WAL-G	physical	yes	yes	delta backups	yes	cloud-first setups, Go shops

The mental model: stanza, repository, WAL archiving

pgBackRest has exactly three concepts you must internalize before touching config. Get these straight and everything else is plumbing.

The stanza is pgBackRest's name for "one PostgreSQL cluster's backup configuration." Everything — backups, archived WAL, retention — is scoped to a stanza. A stanza is identified by name (e.g. main, prod-db) and remembers the cluster's identity: its system identifier (a unique number PostgreSQL generates at initdb time), its version, and its data directory path. This identity-pinning is a feature (it stops you from accidentally backing up the wrong cluster into the wrong repo) and the source of a classic hiccup (after re-initdb or restoring a different cluster into the same path, the stanza check fails — covered in troubleshooting).

The repository is where backups and WAL live — a local path, an NFS mount, or (our case) an S3 bucket. pgBackRest supports up to four simultaneous repositories (repo1 through repo4), which is how you do "local fast restore copy + offsite S3 copy" with a single tool. All repo options are prefixed: repo1-type, repo1-path, repo1-s3-bucket, and so on.

WAL archiving is the continuous half. PostgreSQL, when archive_mode=on, calls your archive_command once per finished 16MB WAL segment. You set that command to pgbackrest archive-push, and pgBackRest ships each segment to the repository. This stream is what makes point-in-time recovery possible — and its failure mode (segments piling up in pg_wal because pushes fail) is the single most common way pgBackRest setups melt down in production.

Step 1 — Install and lay out permissions

Install from PGDG repos (the versions in distro default repos are often ancient — you want ≥2.46 for block incremental, ideally latest):

# Debian/Ubuntu (with PGDG apt repo already configured)
sudo apt-get update && sudo apt-get install -y pgbackrest

# RHEL/Rocky/Alma (with PGDG yum repo)
sudo dnf install -y pgbackrest

# verify — you want 2.50+ in 2026
pgbackrest version

Permissions are the first silent killer. pgBackRest runs as the postgres user (it must read the data directory), so every path it touches must be owned accordingly:

# config file — readable by postgres
sudo mkdir -p /etc/pgbackrest
sudo touch /etc/pgbackrest/pgbackrest.conf
sudo chown postgres:postgres /etc/pgbackrest/pgbackrest.conf
sudo chmod 640 /etc/pgbackrest/pgbackrest.conf

# log and spool dirs
sudo mkdir -p /var/log/pgbackrest /var/spool/pgbackrest
sudo chown postgres:postgres /var/log/pgbackrest /var/spool/pgbackrest
sudo chmod 750 /var/log/pgbackrest /var/spool/pgbackrest

Step 2 — Configure the S3 repository

Here's a complete, production-shaped /etc/pgbackrest/pgbackrest.conf for a single cluster backing up to S3. Every line annotated below:

[global]
# --- repository: S3 ---
repo1-type=s3
repo1-path=/pgbackrest/prod
repo1-s3-bucket=acme-pg-backups
repo1-s3-region=ap-south-1
repo1-s3-endpoint=s3.ap-south-1.amazonaws.com
repo1-s3-key=AKIAXXXXXXXXXXXXXXXX
repo1-s3-key-secret=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

# --- repo encryption (independent of S3 SSE) ---
repo1-cipher-type=aes-256-cbc
repo1-cipher-pass=LONG_RANDOM_PASSPHRASE_FROM_A_SECRET_MANAGER

# --- retention ---
repo1-retention-full=2
repo1-retention-diff=6

# --- performance ---
process-max=4
compress-type=zst
compress-level=3

# --- backup behavior ---
start-fast=y
checksum-page=y
delta=y

# --- logging ---
log-level-console=info
log-level-file=detail
log-path=/var/log/pgbackrest

# --- async archiving (see hidden-configs section) ---
archive-async=y
spool-path=/var/spool/pgbackrest

[prod]
pg1-path=/var/lib/postgresql/16/main
pg1-port=5432
pg1-user=postgres

What each group actually does, and where people trip:

• repo1-path is the prefix inside the bucket, not a filesystem path. Everything pgBackRest writes lands under s3://acme-pg-backups/pgbackrest/prod/. Use distinct paths per environment so a fat-fingered --stanza can't cross-contaminate.
• repo1-s3-endpoint — for AWS, the regional endpoint. For MinIO/Ceph/other S3-compatibles, your gateway host. Two extra options matter for non-AWS: repo1-s3-uri-style=path (MinIO usually needs path-style, i.e. host/bucket/key instead of bucket.host/key) and repo1-storage-verify-tls=n if you're on self-signed certs in a lab (never in prod — install the CA instead, via repo1-storage-ca-file).
• Credentials — static keys in the config work but are the worst option. On EC2/EKS, drop repo1-s3-key/repo1-s3-key-secret entirely and set repo1-s3-key-type=auto — pgBackRest fetches credentials from the instance metadata service / IRSA web identity automatically, no secrets on disk, automatic rotation. This is the single most under-used pgBackRest feature on AWS.
• repo1-cipher-type — client-side AES-256 encryption of everything before it leaves the host. Independent of (and stackable with) S3 server-side encryption. Write the passphrase down somewhere that survives the death of this server — a repo encrypted with a passphrase that only existed in /etc on the machine that just died is a perfectly encrypted pile of nothing.
• compress-type=zst — zstandard: much faster than gzip at similar or better ratios. Default is still gz for compatibility; there's no reason to keep it on a modern system.
• start-fast=y — makes the backup begin immediately by forcing a checkpoint, instead of waiting for the next natural checkpoint (which can be many minutes on a quiet system, leaving you staring at a "hung" backup — a classic confusion).
• checksum-page=y — validates PostgreSQL page checksums during backup, catching silent on-disk corruption at backup time instead of at restore time when it's far too late. Requires the cluster to have been initdb'd with checksums (default in recent Postgres; check with pg_controldata | grep checksum).

The IAM policy the credentials need — least-privilege, per-bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::acme-pg-backups",
      "Condition": {"StringLike": {"s3:prefix": ["pgbackrest/prod/*"]}}
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::acme-pg-backups/pgbackrest/prod/*"
    }
  ]
}

Step 3 — Configure PostgreSQL for archiving

# postgresql.conf
archive_mode = on
archive_command = 'pgbackrest --stanza=prod archive-push %p'
max_wal_senders = 10          # if you also run replicas
wal_level = replica           # minimum for archiving (logical if you need it)

Then restart (not reload — archive_mode requires restart):

sudo systemctl restart postgresql

Three things about archive_command that the docs undersell:

• PostgreSQL only cares about the exit code. Exit 0 = segment archived, safe to recycle. Non-zero = PostgreSQL keeps the segment in pg_wal and retries forever. A misconfigured command doesn't crash anything — it just silently causes pg_wal to grow until the disk fills and the database PANICs. Monitor pg_stat_archiver (see best practices).
• It runs as the postmaster's user with a minimal environment. No login shell, no ~/.aws, no environment variables you set in your shell profile. Everything pgBackRest needs must come from /etc/pgbackrest/pgbackrest.conf. "Works when I run it by hand, fails from PostgreSQL" is almost always an environment difference.
• One segment at a time, synchronously — unless you enable async archiving (below). On a busy write workload, S3's ~50–200ms per-request latency becomes the ceiling on your WAL generation rate. This is why archive-async exists and why you should basically always turn it on with object storage.

Step 4 — Create and verify the stanza

# create the stanza (initializes repo structure in S3)
sudo -u postgres pgbackrest --stanza=prod stanza-create

# verify EVERYTHING: config, archiving round-trip, repo reachability
sudo -u postgres pgbackrest --stanza=prod check

check is the command people skip and then pay for. It forces a WAL switch, waits for that segment to arrive in the repository via your real archive_command, and reads it back. If check passes, your entire pipeline — config file, S3 credentials, bucket policy, archive_command, encryption — is proven working end-to-end. If it fails, you get the actual error now, not during your first 2 AM restore. Run it after every config change and from cron daily.

Step 5 — Full, differential, incremental: how they actually work

pgBackRest has three backup types, and the relationships between them determine both your storage bill and your restore time:

• Full — copies the entire cluster. Self-contained: restoring needs only this backup (plus WAL from during/after it).
• Differential (diff) — copies everything changed since the last full. Restore needs: the full + this one diff. Diffs grow over time (each one contains everything since the full).
• Incremental (incr) — copies everything changed since the last backup of any type. Restore needs: the full + the last diff (if any) + every incr in the chain after it. Small and fast to take, longest dependency chain to restore.

# take each type explicitly
sudo -u postgres pgbackrest --stanza=prod --type=full backup
sudo -u postgres pgbackrest --stanza=prod --type=diff backup
sudo -u postgres pgbackrest --stanza=prod --type=incr backup

# see what you have
sudo -u postgres pgbackrest --stanza=prod info

info output decodes like this — the label encodes the lineage:

full backup: 20260705-020004F
    timestamp start/stop: 2026-07-05 02:00:04+05:30 / 2026-07-05 02:41:33+05:30
    wal start/stop: 000000010000004A00000038 / 000000010000004A0000004F
    database size: 812.4GB, database backup size: 812.4GB
    repo1: backup size: 214.7GB

diff backup: 20260705-020004F_20260708-020003D
    database size: 813.1GB, database backup size: 41.2GB
    repo1: backup size: 9.8GB
    backup reference total: 1 full

incr backup: 20260705-020004F_20260709-020002I
    database size: 813.6GB, database backup size: 6.3GB
    repo1: backup size: 1.4GB
    backup reference total: 1 full, 1 diff

Read the sizes carefully — they answer real questions: database size is the cluster size, database backup size is how much this backup actually copied (41GB of the 813GB changed since the full, for the diff), and repo1 backup size is what it cost in S3 after compression. backup reference total tells you the restore dependency chain explicitly.

Step 6 — Scheduling: cron vs. systemd timers

# /etc/cron.d/pgbackrest  (runs as postgres, mind the user field)
# full every Sunday 02:00, diff Wed 02:00, incr all other nights
0 2 * * 0    postgres  pgbackrest --stanza=prod --type=full backup
0 2 * * 3    postgres  pgbackrest --stanza=prod --type=diff backup
0 2 * * 1,2,4,5,6  postgres  pgbackrest --stanza=prod --type=incr backup

# verify pipeline daily at 08:00 — cheap, catches config drift
0 8 * * *    postgres  pgbackrest --stanza=prod check

Sizing the schedule: the tradeoff is restore complexity vs. backup cost. A common sane default for a database in the hundreds-of-GB range: weekly full, mid-week diff, nightly incr (as above). For multi-TB databases where a full takes most of a day, monthly full + weekly diff + nightly incr — and strongly consider block incremental (below) which changes this math entirely. For small databases (<50GB), just do nightly fulls; the simplicity at restore time is worth more than the storage.

Step 7 — Restore: the part you're actually paid for

Nobody cares about your backups. They care about your restores. The basic full restore, onto a fresh/empty data directory:

# stop postgres, clear the data dir (if it still exists)
sudo systemctl stop postgresql
sudo -u postgres rm -rf /var/lib/postgresql/16/main/*

# restore the latest backup + configure recovery
sudo -u postgres pgbackrest --stanza=prod restore

# start — postgres enters recovery, replays WAL from the repo, promotes
sudo systemctl start postgresql

What restore actually does: picks the latest backup set, walks the dependency chain (full → diff → incrs), copies files back in parallel (process-max applies here too — restores parallelize beautifully), verifies checksums, and writes the recovery configuration into postgresql.auto.conf — including a restore_command = 'pgbackrest --stanza=prod archive-get %f "%p"' so PostgreSQL pulls the WAL it needs from S3 during recovery, and creates recovery.signal. On startup, PostgreSQL replays WAL to the end of the archive and promotes automatically.

Point-in-time recovery

The scenario that justifies the whole setup: someone ran DELETE FROM orders without a WHERE at 14:37. You want the database as it was at 14:36:

sudo systemctl stop postgresql

sudo -u postgres pgbackrest --stanza=prod restore \
  --delta \
  --type=time \
  --target="2026-07-07 14:36:00+05:30" \
  --target-action=pause

sudo systemctl start postgresql

Details that matter and routinely bite:

• Always include the timezone offset in --target. A bare timestamp is interpreted in the server's timezone, and "the DELETE was at 14:37" from a developer's Slack message is usually in their timezone. An hour's ambiguity in either direction defeats the whole exercise. Get the incident time in UTC, write the target in UTC (+00:00).
• --target-action=pause stops recovery at the target and holds the database read-only (recovery paused) instead of promoting immediately. You connect, check SELECT count(*) FROM orders, and only when satisfied run SELECT pg_wal_replay_resume(); to promote. If you overshot or undershot, adjust the target and restore again — promotion is the point of no return: after promotion the cluster starts a new timeline, and replaying further into the old timeline's future requires starting the restore over.
• pgBackRest auto-selects which backup to restore from — the newest backup that ends before your target time. You don't (and shouldn't) pick manually with --set unless you have a specific reason; a backup that ends after the target can't be used, and picking wrongly by hand causes the confusing recovery ended before configured recovery target was reached.
• --type=immediate is the fastest restore: stop at the moment the backup's own consistency is reached, no further WAL replay. Right choice when you just need "a working database from around then" (e.g. seeding a test env), wrong choice for precise incident recovery.
• --type=xid restores to just-before a specific transaction ID — surgical, if you can find the offending xid from logs or pg_waldump.

Delta restore: the underrated one

The --delta flag above deserves its own section. Without it, restore requires an empty data directory — meaning a full re-copy of every byte from S3, which for a terabyte database over a 1Gbps link is 3+ hours of pure transfer before recovery even starts. With --delta, pgBackRest checksums what's already on disk and copies only the files that differ from the backup manifest. When you're restoring onto the same server that just had the incident — where 99% of the data files are perfectly fine — a delta restore finishes in minutes instead of hours. It's also the right way to re-sync a broken replica. The requirement: the data directory must actually be from the same cluster lineage (pgBackRest verifies identity before touching anything, and refuses if the directory looks foreign — a safety check, not a bug).

Troubleshooting: the errors you will actually see

Collected from real-world setups — the exact messages, what they mean, and the fix.

WAL / archiving failures

Symptom / error	Actual cause	Fix
pg_wal filling the disk; pg_stat_archiver shows failed_count climbing	archive_command failing on every segment — S3 creds expired, bucket policy changed, DNS, endpoint typo	Run the push manually as postgres with --log-level-console=detail to see the real S3 error. Fix, then archiving drains the backlog itself. If the disk is minutes from full: temporarily grow the volume or (in true emergency, understanding you're breaking PITR) set archive-push --archive-push-queue-max so pgBackRest drops WAL instead of blocking, then take a fresh full backup immediately after.
WAL segment ... already exists in the archive with a different checksum	Two clusters pushing to the same stanza (a restored clone still has the old archive_command!), or a re-initdb'd cluster reusing segment names	Find the second writer and stop it — the classic culprit is a database cloned from a snapshot for testing, which happily archives into the production repo. Always neuter archive_command (set to /bin/true) on clones before first start.
Archiving is "slow", WAL lag under write bursts	Synchronous, one-at-a-time pushes to S3 latency	archive-async=y + spool-path — batches and parallelizes pushes (see hidden configs). Night-and-day difference on object storage.

Stanza and identity failures

Symptom / error	Actual cause	Fix
backup and archive info files exist but do not match the database	The cluster's system identifier changed — re-initdb, or a different cluster restored into the same path — while the stanza remembers the old identity	If the new cluster is legitimately the one to back up now: stanza-delete (with --force if needed) then stanza-create fresh — old backups for the old cluster are gone with it, so archive them elsewhere first if needed. If this is unexpected: stop — you may be pointing prod config at the wrong data directory.
stanza-upgrade forgotten after a major version upgrade; backups fail with version mismatch	Stanza pins PG version; pg_upgrade changed it	pgbackrest --stanza=prod stanza-upgrade, then immediately a fresh --type=full backup (old-version backups can't base new-version incrementals).
unable to acquire lock ... process ... already running	Another backup/expire genuinely running, or a stale lock from a killed process	Check ps; if genuinely stale, remove the lock file under the lock path (default /tmp/pgbackrest). Consider moving lock-path off /tmp if your distro aggressively cleans it mid-backup.

S3-specific failures

Symptom / error	Actual cause	Fix
HTTP error 403 on any operation	Wrong credentials, clock skew (S3 signatures are time-sensitive!), or bucket policy missing an action	Check NTP first — a host clock more than ~15 min off fails all SigV4 auth with 403, and this one genuinely mystifies people. Then verify the IAM policy has ListBucket + Get/Put/DeleteObject on the right prefix.
unable to verify TLS certificate	Self-signed or private-CA cert on MinIO/Ceph endpoint	Point repo1-storage-ca-file at your CA bundle. repo1-storage-verify-tls=n only for throwaway labs.
Connection succeeds but bucket operations 404 / NoSuchBucket on MinIO	Virtual-host-style URLs against a path-style-only gateway	repo1-s3-uri-style=path.
Backups randomly fail mid-transfer on flaky networks	Long S3 transfers hitting transient resets	pgBackRest retries internally; if it still fails, raise io-timeout, and check for an overloaded NAT gateway — instance-level S3 VPC endpoints remove that whole failure class (and the NAT data-processing bill).

Restore failures

Symptom / error	Actual cause	Fix
unable to find ... in the archive during recovery	Recovery needs a WAL segment that never got archived (archiving was broken during that window) or was expired	If the segment truly never made it, PITR through that gap is impossible — restore to just before the gap (--type=time with an earlier target) or from a later backup past it. This is why you alert on archiver failures: gaps are only discoverable at restore time otherwise. pgbackrest verify can audit the archive proactively.
recovery ended before configured recovery target was reached	Target time is after the end of available WAL, or you forced an unsuitable backup with --set	Check pgbackrest info for the max archived WAL; let auto-selection pick the backup; confirm the target's timezone math.
Restore succeeds, postgres starts, but immediately promotes without honoring the target	Leftover postgresql.auto.conf recovery settings from a previous attempt, or missing recovery.signal	Let pgBackRest write recovery config itself; don't hand-edit between attempts. Between retries, clear stale signal files and auto.conf recovery lines (a fresh restore does this for you).
Restore is slow despite big process-max	Single-stream bottleneck elsewhere: compression, one giant table file, EBS throughput cap	zstd decompression is rarely the issue; check cloud volume throughput limits (a gp3 at default 125MB/s caps a 1TB restore at ~2.5h no matter the parallelism). Provision restore-time IOPS/throughput before the disaster, not during.

Hidden and under-used configs worth knowing

Everything here is documented, but you'd never find it unless you read the release notes for years. These are the settings that most upgrade a default setup:

1. Block incremental — repo1-block=y (2.46+)

Standard incrementals work at file granularity: one row updated in a 1GB table file means the whole 1GB file is recopied. Block incremental splits files into (size-tuned) blocks and stores only changed blocks, using a block map to reconstruct at restore. For big, hot tables with scattered writes — the normal OLTP shape — this can cut incremental sizes by an order of magnitude. Combine with repo1-bundle=y (required pairing) and it's the single biggest storage/time win available in modern pgBackRest:

repo1-bundle=y
repo1-block=y

2. File bundling — repo1-bundle=y

PostgreSQL clusters contain thousands of tiny files (small relations, forks, FSMs). Backing each up as its own S3 object means thousands of PUT requests (billed each!) and hideous small-object throughput. Bundling packs small files together into larger objects — fewer requests, faster backups, lower bill. Tunables: repo1-bundle-size (target bundle size, default 20MiB) and repo1-bundle-limit (files larger than this stay standalone, default 2MiB). There's no real downside on object storage; it's off by default only for backward compatibility.

3. Async archiving — archive-async=y + spool-path

The default archive path is synchronous: PostgreSQL hands over one segment, pgBackRest pushes it to S3, returns. At ~100ms+ per S3 round trip, that's a hard ceiling of ~10 segments/sec — a busy cluster generates more. Async mode decouples: archive-push acknowledges segments quickly into a local spool, while a background process pushes batches to S3 in parallel (process-max applies). Also works on the restore side: archive-get prefetches upcoming WAL segments into the spool during recovery, dramatically speeding PITR WAL replay from S3. The spool directory needs to survive being lost (it's transient state), but put it on real disk, not tmpfs — after a crash, tmpfs contents vanish and archiving state resets unnecessarily.

4. expire-auto and manual expiry control

By default, expiry runs after each successful backup. expire-auto=n decouples it, so you can run pgbackrest expire on its own schedule — useful when expiry (which deletes potentially many S3 objects) is slow and you don't want it inside your backup window, or when you want a human/automation gate before anything is deleted.

5. Backup from standby — backup-standby=y

With pg2-* options pointing at a standby, pgBackRest copies the data files from the standby (offloading all read I/O from the primary), while still coordinating the backup start/stop against the primary. The standby needs the same pgBackRest config and repo access. Caveat: the standby must be reasonably caught up, or the backup waits.

6. Multi-repository — local + offsite in one config

# repo1: local NFS for fast restores
repo1-type=posix
repo1-path=/backup/pgbackrest
repo1-retention-full=2

# repo2: S3 for disaster recovery
repo2-type=s3
repo2-s3-bucket=acme-pg-backups
repo2-retention-full=4
# ... s3 options ...

Archiving pushes WAL to both automatically. Backups target one repo per run (--repo=2), so schedule them separately. Restores prefer repo1 by default (fast local path) with S3 as fallback via --repo. This is the textbook 3-2-1 layout with a single tool.

7. Miscellaneous sharp tools

• archive-push-queue-max — hard cap on how much WAL may queue locally when the repo is unreachable; beyond it, pgBackRest drops WAL (accepting a broken PITR chain) rather than letting the disk fill and crash the database. A deliberate "lose recoverability, keep availability" tradeoff — set it only if that's genuinely your preference, and pair it with alerting so a dropped-WAL event triggers an immediate full backup.
• db-exclude — skip specific databases within the cluster during restore (restore is still cluster-level; excluded DBs come back as unusable shells to be dropped).
• --target-timeline — after multiple restore/promote cycles you get a timeline tree; this picks which branch to follow. If you've restored more than once during an incident, learn timelines before the third attempt, not after.
• tablespace-map — remap tablespace locations at restore time, essential when the target machine's mount layout differs from the source.
• log-level-file=detail — keep file logging at detail permanently; disk is cheap and the detailed log is what you'll want during any incident post-mortem.
• pgbackrest verify — audits repository integrity: re-checksums backup files and archived WAL in the repo, catching bit-rot or partial uploads before restore time. Run monthly via cron on a machine that isn't the primary.

Production best practices

FAQ

Can I back up to S3 without a dedicated backup server?

Yes — the simplest production-legitimate topology is exactly what this guide builds: pgBackRest on the database host itself, repository in S3. A dedicated repository host (pgBackRest running remotely, connecting over TLS/SSH) adds value when you have many clusters to centralize, want backups to keep working while a DB host is compromised/degraded, or need to offload compression CPU — but it is not a requirement for a solid setup.

Does taking a backup slow down my database?

Some — backup reads compete for disk I/O and compression uses CPU, bounded by process-max. Mitigations in order: run in the low-traffic window, lower process-max, use backup-standby=y to shift all read I/O to a replica. WAL archiving overhead, by contrast, is continuous but tiny (one small process per segment or an async batch).

Full weekly + daily incr, or full weekly + daily diff?

Diffs make restores simpler (max chain: full + one diff) but each diff re-copies everything since the full, so by day 6 they're big. Incrs are minimal daily cost but the restore chain walks every one of them, and one corrupted/expired link breaks everything after it. The hybrid in this guide (weekly full, mid-week diff, nightly incr) caps the chain at ~4 while keeping nightly cost low. With repo1-block=y, incrementals get so cheap that many teams just do full + nightly block-incrementals and stop thinking about diffs entirely.

How do I restore just one database out of the cluster?

Physical backups are cluster-level — you can't extract one database directly. The pattern: restore the cluster (or a PITR of it) onto a scratch machine with --db-include (which restores only the named database's files plus the system catalogs, much faster), then pg_dump the recovered database from the scratch instance into production. Keep a scratch-restore runbook for exactly this — "someone dropped one table" is far more common than full-cluster disasters.

Does pgBackRest encryption make S3 server-side encryption redundant?

They protect against different threats. repo1-cipher-type (client-side) means AWS/the storage operator never sees plaintext and a leaked bucket is useless without your passphrase. SSE (server-side) protects disks in the provider's datacenter but anyone with bucket read access gets plaintext. Client-side is the one that matters for your threat model; enabling both costs nothing.

What happens to my backups when I upgrade PostgreSQL?

After pg_upgrade, run pgbackrest stanza-upgrade, then take an immediate full backup. Old-version backups remain restorable (into the old version's binaries!) until retention expires them, but they cannot serve as a base for new-version incrementals — hence the mandatory fresh full. Forgetting stanza-upgrade is a top-5 post-upgrade ticket.

Can pgBackRest seed a streaming replica?

Yes, and it's the best way: pgbackrest restore on the replica host with --type=standby writes standby.signal and the recovery settings; the replica replays WAL from the repo (fast, parallel, doesn't load the primary) and then connects to the primary for streaming. Re-syncing a diverged replica with --delta is similarly much cheaper than a fresh pg_basebackup.

How much S3 storage will I actually use?

Rule of thumb with zstd: fulls land at ~25–40% of database size (very data-dependent), diffs/incrs proportional to churn, WAL at churn rate. With repo1-retention-full=2, weekly fulls, and modest churn, budget roughly the database size in S3. Block incremental + bundling can cut that substantially. Watch the request costs too, not just storage — that's what bundling fixes.

Is there a dry-run for expiry?

Yes: pgbackrest --stanza=prod --dry-run expire shows exactly what would be deleted. Run it before changing any retention setting — retention mistakes are the one error category with no undo.

Glacier / cold storage for old backups?

Not directly through pgBackRest — it needs read access to everything it manages, and Glacier retrieval latency breaks that. The workable patterns: S3 lifecycle rules transitioning objects to Infrequent Access (works transparently, cheaper, still instant reads), or a second repo whose bucket you archive by external means, accepting those backups are "break glass, hours to thaw." Never lifecycle-transition the archive/ (WAL) path of an active stanza to Glacier — recovery reads from it.

Takeaways

• pg_dump is an export, not disaster recovery. Production DR is physical backup + continuous WAL archiving, and pgBackRest is the strongest open-source orchestrator of that pattern.
• Three concepts carry everything: the stanza (per-cluster identity), the repository (S3-native, up to 4 of them), and the WAL stream (whose silent failure is the #1 production incident).
• Full / diff / incr trade backup cost against restore-chain length — and incrementals silently upgrade to fulls when their base disappears, so alert on duration, not just success.
• The restore flags that matter: --delta (reuse intact files — hours to minutes), --type=time --target-action=pause (PITR with a verify-before-promote gate), timezone-explicit targets always.
• Enable the modern trio almost nobody defaults to: repo1-bundle=y, repo1-block=y, archive-async=y — fewer S3 requests, order-of-magnitude smaller incrementals, archiving that keeps up with write bursts.
• The cipher passphrase must outlive the server. Encrypted repo + passphrase only on the dead host = no backups at all.
• Monitor pg_stat_archiver and pg_wal size — the continuous failure modes that success/failure backup alerts never catch.
• A backup you haven't restored is a hypothesis. Quarterly blank-machine drills, monthly verify, daily check.

References & extra reads

• pgBackRest User Guide — the canonical walkthrough; genuinely good, read it end to end once.
• pgBackRest Configuration Reference — every option, including the hidden ones covered here.
• pgBackRest Command Reference — per-command flags: restore targets, expire dry-run, verify.
• PostgreSQL docs — Continuous Archiving and PITR — the underlying mechanism pgBackRest orchestrates; understand this and every pgBackRest behavior makes sense.
• PostgreSQL docs — pg_stat_archiver — the view to alert on.
• pgBackRest release notes — where features like block incremental and bundling first appear; worth skimming at every upgrade.
• pgstef's blog (Stefan Fercot) — pgBackRest maintainer-adjacent deep dives: multi-repo, standby backups, PITR walkthroughs.
• cvam.sight — Debugging Database Issues — slow queries, locks, replication lag — the operational other half of running PostgreSQL.