Cutting the AWS Bill — 28 case studies

Part 3: Storage and Data Lifecycle

Part 3 of 10 · intermediate

Sep 26, 2026 · devops · 17 min read · 3320 words intermediate

Storage lifecycle: when bytes pay the bill and objects pay the migration.

devops aws s3 storage

Storage bills grow quietly because data rarely gets deleted. Three case studies show how to match storage class to real access patterns and, crucially, how to price the migration itself: Canva saved $3.6M a year by moving user content to S3 Glacier Instant Retrieval (paying about $1.6M once to transition roughly 80 billion objects), Snap moved more than two exabytes and saved tens of millions, and Rewind's lifecycle-rule edit came with a $103K invoice and a six-month payback. The unifying insight is unit economics applied to storage: ongoing savings scale with bytes, the one-off transition cost scales with object count, and their ratio, the average object size, is the best single predictor of payback. This part builds that rule into a worked break-even table and a complete S3 and EBS lifecycle checklist.

Get the full report. This series expands the research report Cutting the AWS Bill: 28 Case Studies (53 pages, PDF, 780 KB). One click starts the download.

⬇ Download PDF

1. Why storage quietly dominates

Compute is visible: instances, tasks and nodes appear on dashboards and in on-call rotations. Storage is not. It accumulates in the background as backups, log archives, container images, snapshots, object versions and user uploads, and nobody ever decides that it should keep growing. Storage cost is bytes times copies times duration, and all three multipliers rise on their own.

The three cases in this part come from very different businesses (an online design tool, a social network, a backup provider) and share one method: measure how the data is actually accessed, choose the cheapest class that keeps the product promise, and do the payback maths for the migration before flipping the switch. The last step is the one teams forget, and it is where Rewind's story lives.

Two different meters run when you change storage class Ongoing saving scales with BYTES (GB x price difference x months) paid back every month One-off transition cost scales with OBJECT COUNT (~$0.02 per 1,000 requests) paid once, up front Ratio of the two = average object size = payback predictor

Fig. 1 - Storage economics in one picture: bytes save money, objects cost money.

2. The S3 storage classes and their prices

The report uses approximate us-east-1 list prices as of September 2026; always confirm on the current AWS pricing pages. The numbers below are the ones that drive every calculation in this part.

ClassApprox. price per GB-monthAccessWatch for
S3 Standard$0.023MillisecondsThe default; nothing to migrate
Standard-IA$0.0125Milliseconds30-day minimum, 128 KB minimum billable size, retrieval fee
Glacier Instant Retrieval$0.004Milliseconds90-day minimum, 128 KB minimum size, higher retrieval charge than IA
Glacier Deep Archive$0.00099Hours180-day minimum; only for data that must be kept but is almost never read
Intelligent-TieringAutomaticMilliseconds (frequent/infrequent tiers)Per-object monitoring fee; objects under 128 KB are not auto-tiered

Transitions between classes are billed per request, at roughly $0.02 per thousand objects for the classes discussed here. That per-object charge, set against a per-byte saving, is the whole story of this part.

3. CS-07 Canva: bytes pay the bill, objects pay the migration

FieldDetail
BusinessOnline design tool, 100M+ monthly active users
Data230+ PB in S3; largest bucket ~45 PB
ChangeStandard-IA to S3 Glacier Instant Retrieval
Result$3.6M saved per year (~$300K a month), growing with data

3.1 Context

Canva keeps templates, stock photos and graphics in S3 Standard because users hit them constantly. User uploads and designs behave differently: they are accessed heavily for a short time while a design is being made, then rarely. Those had historically lived in Standard-IA, while logs and backups used Glacier Flexible Retrieval.

3.2 What they did

  • Measured access patterns. Using S3 Storage Class Analysis and internal frequency-segmentation work, Canva saw retrievals drop sharply after about 15 days, while the IA data kept a flat, low retrieval rate over time.
  • Waited for the right product. S3 Glacier Instant Retrieval, launched in late 2021, offered archive-level storage prices with millisecond access, a fit for user content that is rarely opened but must open instantly.
  • Did the migration maths per bucket. Transitions are billed per object, while savings accrue per byte. Moving everything would have cost millions (a third-party write-up puts it near $6M). So Canva computed break-even time by each bucket's average object size and prioritised buckets whose average object was roughly 400 KB or larger.
  • Moved selectively. About 80 billion objects were transitioned, for a one-off cost of about $1.6M.
Result. Roughly 130 PB of the 230 PB now sits in Glacier Instant Retrieval. About $300K saved per month ($3.6M a year), with positive ROI after only a few months. The saving keeps growing because user-generated data keeps growing.

3.3 The thinking pattern

This is unit economics applied to storage. Ongoing savings scale with bytes; one-time transition cost scales with object count. Their ratio is the average object size, which makes it the single best predictor of payback. Rank buckets by that ratio and migrate the fastest-payback buckets first. Canva's own payback, per the report's arithmetic, is $1.6M divided by $0.3M a month, about 5.3 months.

Lessons to steal.

  • Measure access before choosing a class: S3 Storage Class Analysis and Storage Lens are cheap ways to do it.
  • Compute payback per bucket (section 5 has a ready-made table).
  • Glacier IR has a 90-day minimum storage duration and a 128 KB minimum billable object size, and its retrievals cost more than IA.
  • Re-check your storage map whenever AWS launches a new class. New price points change the optimal answer.
Watch-out. Glacier IR is only a win if retrievals really are rare. A product change that makes old content popular again can reverse the economics, so keep monitoring retrieval volume.

4. CS-08 Snap Inc.: two exabytes of Memories on Glacier Instant Retrieval

FieldDetail
BusinessSnapchat, 363M+ daily active users at the time
Data2+ exabytes, about 1.5 trillion photos and videos
ChangeLong-lived media to S3 Glacier Instant Retrieval
ResultTens of millions of dollars saved, same user experience

Snapchat began as an ephemeral app, but features like Memories turned it into a long-term media store. Most saved media is rarely re-opened, yet it must load instantly when it is. Snap migrated its storage from another cloud provider to S3 and placed long-term, rarely accessed media in S3 Glacier Instant Retrieval. More than 2 EB (roughly 1.5 trillion files) moved without users noticing.

Result. Tens of millions of dollars saved, 99.99% availability, and 20-30% lower download latency in some regions.

The thinking pattern. Start from the product promise (instant access, rare access) and pick the cheapest storage that keeps that promise. Lessons: translate product features into access SLOs, so the SLO rather than habit chooses the storage class; at exabyte scale, even a few tenths of a cent per GB-month is tens of millions of dollars a year.

Watch-out. Snap's move also involved a cross-cloud migration; its economics reflect negotiated pricing at enormous scale. Do not expect the same percentages at a smaller footprint.

5. CS-09 Rewind: a lifecycle-rule edit with a $103K invoice attached

FieldDetail
BusinessSaaS backup provider
DataTens of billions of small objects
ChangeStandard-IA to Glacier IR via a lifecycle rule
Result~35% lower total S3 cost, ~6-month payback

A backup company stores huge numbers of objects that are almost never read, which makes it a textbook Glacier IR candidate. But Rewind's objects are small, and that changes everything.

  • Priced the transition first. Moving the content would cost about $103,000 in one-off transition requests against roughly $16,000 of monthly savings, a payback of a little over six months. They accepted it and amortised the cost.
  • Respected the 90-day minimum. Glacier IR bills at least 90 days of storage per object. The lifecycle transition was set at 90 days so objects deleted earlier would not incur early-deletion charges.
  • Let S3 do the work. After the rule change, S3 transitioned the eligible objects in about two days.
Result. Overall S3 cost dropped about 35% while stored bytes stayed flat.

The thinking pattern. A single configuration change can produce a large one-time bill. Estimate the one-off cost and the payback period before you save the lifecycle rule, then decide deliberately. Lessons: payback equals one-off cost divided by monthly saving (write it down before you flip the switch); align transition ages with minimum storage durations to avoid early-deletion fees; small objects are expensive to move, so consider aggregating them before archiving.

Watch-out. Rewind's objects were small, which made the transition expensive relative to savings. With larger objects the payback would have been much faster; compare Canva.

6. The break-even formula, worked

The report distils the Canva and Rewind experience into one formula. Using approximate list prices (Standard-IA $0.0125 and Glacier IR $0.004 per GB-month, about $0.02 per 1,000 transition requests), the number of months to break even for an average object of a given size is:

months = (0.02 / 1000) / (object_GB x (0.0125 - 0.004))

# per-object transition cost: $0.00002
# per-object monthly saving: object_GB x $0.0085
Average object sizeMonths to break evenVerdict
128 KB19.3Only if data will stay cold for years
200 KB12.3Marginal
400 KB6.2Canva's cut-off; good
1 MB2.4Excellent
4 MB0.6Immediate win

You can verify one row by hand. For a 400 KB object, the object size in GB is about 0.0004. The monthly saving is 0.0004 times $0.0085, which is $0.0000034. The one-off transition cost is $0.00002. Divide: about 5.9 months with decimal gigabytes, or 6.2 months if you use binary gigabytes (1 GB = 1,048,576 KB), which is the convention behind the table. The point is not the second digit but the shape: doubling the object size halves the payback, and below a couple of hundred kilobytes the migration barely pays.

Include retrieval volume if data is not truly cold. Glacier IR bills a minimum of 90 days per object and 128 KB per object, and its retrieval charges are higher than Standard-IA's. If a meaningful share of the data is read each month, the retrieval fees can erase the storage saving. Canva's whole case rests on retrievals really being rare after about 15 days.

6.1 A worked example with your own numbers

Suppose a bucket holds 500 TB of images averaging 800 KB each in Standard-IA. The object count is 500 TB divided by 800 KB, roughly 650 million objects. The one-off transition cost is 650 million divided by 1,000, times $0.02, which is about $13,000. The monthly saving is 500,000 GB times ($0.0125 minus $0.004), about $4,250. Payback is about three months. By contrast, a 500 TB bucket of 30 KB log files holds around 17 billion objects; the transition would cost roughly $350,000 against the same $4,250 monthly saving, an 80-month payback, and objects under 128 KB are billed as 128 KB anyway. Same bytes, opposite decision.

7. Turning the lessons into lifecycle rules

Every storage case ends in a lifecycle configuration. A minimal but complete rule set for a versioned bucket does five things: transitions cold data, expires noncurrent versions, aborts incomplete multipart uploads, and (where appropriate) expires data outright. The JSON below is a sketch of the S3 lifecycle configuration structure; adapt the prefixes, ages and classes to your measured access data, and remember the 90-day minimum when choosing the transition age for Glacier IR.

{
  "Rules": [
    {
      "ID": "cold-after-90d",
      "Status": "Enabled",
      "Filter": { "Prefix": "user-content/" },
      "Transitions": [
        { "Days": 90, "StorageClass": "GLACIER_IR" }
      ]
    },
    {
      "ID": "expire-noncurrent",
      "Status": "Enabled",
      "Filter": {},
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30 }
    },
    {
      "ID": "abort-incomplete-mpu",
      "Status": "Enabled",
      "Filter": {},
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

Three details from the cases matter more than the syntax. First, pause event consumers before a bulk transition: Levels.fyi's S3 event notifications fed an SQS pipeline, and a mass storage-class change would have flooded the backend, so the listeners were detached for the migration and re-attached afterwards. Second, versioned buckets keep every overwrite forever unless you expire noncurrent versions, which is exactly the Duolingo finding. Third, incomplete multipart uploads are billed but invisible in listings, so add the abort rule to every bucket.

8. Beyond S3: EBS, snapshots, registries and backups

The same bytes-times-copies-times-duration logic applies outside S3, and the earlier cases supply concrete moves:

  • EBS gp2 to gp3. About 20% cheaper per GB, with IOPS and throughput decoupled from volume size. Levels.fyi did this for RDS after backups, in a low-traffic window.
  • Unattached volumes and old snapshots. Orphans accumulate after instances are deleted. Snapshot before deleting.
  • Tiered backup retention. Grandfather-father-son rotation (daily for a month, weekly for a year, monthly long-term) keeps recovery points while cutting copies; Levels.fyi cut RDS backup storage from 15.7 TB to 3 TB.
  • ECR lifecycle policies. Old images pile up per build; keep the last N (Levels.fyi kept 25) plus anything referenced by a running or rollback release.
  • DynamoDB. TTL for automatic deletion (backfill the attribute for existing rows), and the Standard-IA table class for large, rarely read tables.
PostgreSQL on S3 with pgBackRest. If your backups live in an S3 repository, write down the retention you actually need (full, differential and WAL) and configure pgBackRest expiry to match. Be careful with lifecycle transitions on the repository prefix: objects that pgBackRest expires early may incur minimum-duration charges in IA or Glacier classes. Do the payback maths first (Part 9 returns to this).

9. A decision flow for any bucket

  1. Measure. Turn on Storage Class Analysis or Storage Lens and collect at least 30 days of access data per prefix.
  2. Segment. Group prefixes by average object size and by age-versus-retrieval curve. Canva's curve showed retrievals collapsing after roughly 15 days.
  3. Price the move. Compute per-bucket payback with the formula in section 6. Discard anything with payback beyond your tolerance (the report treats under six months as an easy yes and above eighteen as needing a strategic reason).
  4. Protect the pipeline. Identify event consumers, replication rules and analytics jobs that a bulk transition would trigger, and pause or filter them.
  5. Respect minimums. Set lifecycle ages at or beyond the target class's minimum storage duration.
  6. Roll out by bucket, fastest payback first. Then watch retrieval volume monthly; a popular-again product feature can reverse the economics.

10. Measuring object size and age with S3 Inventory

The break-even formula needs two inputs per bucket or prefix: average object size and how quickly objects go cold. You can get both without touching production traffic. Enable S3 Inventory on the bucket (daily or weekly, Parquet output), which lists every object with its size, storage class and last-modified date, then query it with Athena. The sketch below groups a bucket by top-level prefix and computes count, total size and average object size; the column names follow the standard inventory schema, so confirm them against your own configuration.

SELECT split_part(key, '/', 1)                    AS prefix,
       COUNT(*)                                   AS objects,
       ROUND(SUM(size) / POWER(1024, 4), 2)       AS total_tb,
       ROUND(AVG(size) / 1024, 1)                 AS avg_kb,
       ROUND(SUM(CASE WHEN size < 131072 THEN 1 ELSE 0 END)
             * 100.0 / COUNT(*), 1)               AS pct_under_128kb
FROM   s3_inventory
WHERE  is_latest = TRUE
GROUP  BY 1
ORDER  BY total_tb DESC
LIMIT  25;

The last column matters because objects under 128 KB are billed as 128 KB in the IA and Glacier IR classes and are skipped by Intelligent-Tiering. A prefix where most objects fall under that threshold is a candidate for aggregation, expiry or simply staying put, not for a class transition. For age, group by the month of last_modified_date and compare against Storage Class Analysis output: the curve you want is bytes retrieved per byte stored, by object age. Canva's curve fell off a cliff around 15 days; yours may not.

10.1 Retrieval sensitivity: when "cold" is not cold enough

The break-even table assumes retrievals are negligible. Before committing, model the sensitivity. If a class charges a per-GB retrieval fee, then a fraction r of the data read each month adds r times the retrieval price to the monthly cost. For a bucket where a few percent is read monthly the extra cost is small next to the storage saving; for a bucket where a third is read monthly it can exceed the saving. A practical rule from the cases: only tier data whose monthly retrieval volume you have measured, over a full business cycle including any seasonal peak, and keep an alert on retrieval bytes after the move so a product change cannot silently reverse the decision.

11. Common storage failure modes

Failure modeWhat happensGuard rail
Bulk transition fires eventsS3 notifications flood SQS or Lambda consumers (Levels.fyi)Detach or filter listeners; re-attach afterwards
Small-object archiveTransition cost dwarfs saving (Rewind: $103K for $16K a month)Compute per-prefix payback; aggregate small objects first
Transition age below minimumEarly-deletion charges on objects removed before 90 daysSet transition age at or above the class minimum
Versioning left unboundedEvery overwrite kept forever (Duolingo)Expire noncurrent versions after a defined window
Incomplete multipart uploadsBilled parts nobody can seeAbort rule on every bucket
Product reheats cold dataRetrieval fees exceed the saving (Canva watch-out)Alert on retrieval bytes; review after launches
New TTL assumed to clean old rowsDynamoDB rows written before the TTL persist (Duolingo)Backfill the TTL attribute or run a one-off purge
Maximal compliance retentionBackups grow far past database size (Levels.fyi 15.7 TB)Define, document and follow a tiered policy

Almost every row is an old decision that was correct once. The remedy is procedural as much as technical: give every bucket an owner, a retention statement and a review date, and put lifecycle configuration in the same Terraform module that creates the bucket so a new bucket cannot exist without one.

11.1 Ownership is the hard part

Deleting data is a business decision, not an infrastructure one, which is why storage waste persists. Canva could act because it had frequency-segmentation data and a clear product owner for user content. Snap could act because the product promise was explicit. Where nobody owns a bucket, nobody will approve a deletion or a class change, and the bytes accumulate. A lightweight fix that appears in several of the cases is a one-page retention table: what the data is, how many copies exist, how long each copy is kept, why, and who can change it. It takes an afternoon and removes most of the debate the next time someone asks whether a bucket can be tiered or trimmed.

Why not just use Intelligent-Tiering everywhere?

It is a good fit where access is unpredictable and objects are larger, as Airbnb uses it. But it charges a per-object monitoring fee and does not auto-tier objects under 128 KB, so buckets full of small objects gain little. Where access is predictable (Canva measured a clear 15-day cliff) an explicit lifecycle rule is cheaper.

What if my average object size is small?

The Rewind case is the answer: small objects make transitions expensive relative to the saving, so payback stretches. Consider aggregating small objects into larger archives before tiering, or accept the longer payback only if the data will stay cold for years.

Is Glacier Instant Retrieval always cheaper than Standard-IA?

Only when retrievals are rare. Storage is much cheaper ($0.004 versus $0.0125 per GB-month) but retrieval charges are higher and there is a 90-day minimum. Monitor retrieval volume after the move.

Takeaways

  • Bytes save money; objects cost money. Their ratio, the average object size, predicts payback better than anything else.
  • Measure access first with Storage Class Analysis or Storage Lens; Canva saw retrievals collapse after ~15 days.
  • Price the one-off cost before saving a lifecycle rule. Rewind's rule edit came with a $103K invoice and a six-month payback.
  • Honour minimum durations and sizes (90 days and 128 KB for Glacier IR) or pay early-deletion and rounding charges.
  • Pause event consumers before bulk transitions; expire noncurrent versions; abort incomplete multipart uploads.
  • Start from the product promise. Snap chose the cheapest class that kept "instant but rare".

References & further reading

Keep it offline. The complete 28-case report, with the scoreboard, lever matrix and 90-day playbook, is one PDF.

⬇ Download PDF
← Part 2 next: Part 4 →
© cvam — written in plaintext, served warm