Visibility first: six teams that read their AWS bill line by line.

Six of the 28 case studies share a method more than a technology: each team turned billing data into something it could query, ranked what it found, and asked of every large number, "is this expected, or is it a bug nobody noticed?" The results run from a $2,410-a-month clean-up at Levels.fyi (twice), to Segment's $1M-a-year push, Duolingo's 20% reduction with a five-person FinOps team, Lyft's 40% drop in AWS cost per ride, and Airbnb's decision to treat billing as a data product. This part retells each case in detail, extracts the thinking patterns (verify from the inside before deleting, compliance means defined retention not maximal retention, internal traffic must never pay per-request edge prices), and ends with a practical, copy-ready way to start.

1. Why visibility comes first

Every other part of this series, from storage tiering to Graviton to leaving AWS entirely, depends on the habit these six cases demonstrate. The savings range from a few thousand dollars a month to over a million a year, but the thinking is identical: turn billing data into something you can query, rank what you find, and question whether each big number is expected. It is the foundation every later chapter builds on.

The chapter title says "hygiene" for a reason. Almost nothing in these stories was a bug. The systems were doing exactly what someone configured years earlier: backups kept for compliance, logs shipped to premium destinations, autoscaling written for old traffic, capacity raised during an incident and never lowered. Cost reviews are really reviews of old decisions.

2. CS-01 Levels.fyi, Part 1 (2024): an inverted treasure hunt

Field	Detail
Business	Compensation-data platform, ~650M page views and ~26M visitors in 12 months
Stack	CloudFront, ECS Fargate, RDS, Lambda, ElastiCache, S3
Published	June 2024
Result	~15% of total cloud spend removed

2.1 Context

Levels.fyi runs a lean, AWS-native stack. When the team opened Cost Explorer they found roughly a quarter of the bill was CloudFront, with the rest spread across compute, databases, caches and storage. Nothing was on fire; the goal was simply to stop paying for things that no longer earned their keep. The write-up calls it an "inverted treasure hunt": instead of looking for gold, they looked for things to bury.

2.2 What they did

• Deleted forgotten resources. EC2 instances that had been spun up temporarily so employees could reach internal systems were still running long after the need passed. They were removed together with their attached EBS volumes.
• Verified before deleting. An ElastiCache cluster was referenced by no service, yet CloudWatch still showed CPU and network activity. Instead of trusting the graph, the team connected to it and checked DBSIZE and INFO (connected clients, total commands processed). The "activity" turned out to be the backup job. One final backup was taken, then the cluster was deleted.
• Tiered cold data. Terabytes of historical job data in S3 moved to Glacier Deep Archive with a lifecycle rule. Because S3 event notifications fed an SQS pipeline, a mass storage-class change would have flooded the backend with events, so the listeners were detached for the migration and re-attached afterwards.
• Re-tuned autoscaling. Fargate scaling rules written for an older traffic shape kept several tasks alive during quiet periods; they were re-fitted to current traffic.
• Moved to newer, cheaper hardware. RDS went from t3 to Graviton-based t4g instances and from gp2 to gp3 volumes (after backups, in a low-traffic window). Lambda functions moved to arm64 after dependency compatibility checks.
• Retention hygiene. An ECR lifecycle rule now keeps only the last 25 image versions, CloudWatch log retention was shortened, and load balancers and NAT gateways were audited for anything unused.
• Commitments. Reservations plus a third-party group-buying service trimmed roughly $700 a month from EC2 and ECS.

2.3 Results and pattern

The thinking pattern. Start from the bill, not the architecture diagram. Treat every line item as a claim that must be proven ("this resource is needed"), and prove it from inside the resource rather than from surface metrics. Before changing anything wired into an event-driven system, ask what side effects the change itself will emit.

Lessons to steal.

• Anything created "temporarily" needs an owner tag and an expiry date, or it becomes permanent spend.
• Activity is not usage: backups, health checks and replication generate metrics on resources nobody uses.
• Storage-class transitions can fire S3 events; pause or filter consumers before bulk transitions.
• Graviton instance classes and gp3 volumes are low-risk, high-certainty wins for managed databases.

3. CS-02 Levels.fyi, Part 2 (September 2026): four fixes, $28.9K a year, nothing switched off

Two years after the first clean-up, the team returned with a different question. Instead of hunting for idle resources, they took each large line item in Cost Explorer and asked: is this number expected, or does something feel wrong? To move fast, they gave an LLM read-only AWS credentials and let it query Cost Explorer, CloudWatch and service APIs through the AWS CLI, then pointed it at questions with measurable answers: which log groups were growing fastest and what was writing to them, or what was pushing RDS cost up by a small amount every day.

3.1 Fix one: backups from 15.7 TB to 3 TB

SOC 2 preparation had produced daily RDS snapshots on a long, flat retention. Backup storage had quietly become most of the RDS bill, far exceeding the size of the databases themselves. Re-reading the requirement showed SOC 2 asks for a defined retention period that you actually follow, not a maximal one. A tiered policy (daily for a month, weekly for a year, monthly long-term, a few long-term copies) cut storage from 15.7 TB to 3 TB and saved about $1,200 a month while keeping point-in-time recovery for recent days.

The report includes a sanity check worth repeating. At roughly $0.095 per GB-month for RDS backup storage beyond the free allowance, 12.7 TB removed is about $1,235 a month, which matches the reported saving. Reproducing a claimed saving from list prices is the fastest test that you understand where the money went.

3.2 Fix two: WAF and CDN logs from $640 to under $70

Logs were shipped to one of AWS's most expensive destinations (priced well above S3) and rarely read, and they included every allowed request. Moving them to cheaper storage and keeping only the events actually reviewed, chiefly blocked traffic, saved about $600 a month.

3.3 Fix three: one line of logging

A small line item kept creeping upward: $26, then $36, then $41. The cause was a single statement in the shared Lambda entry point that logged the entire incoming event on every invocation. A compact summary line replaced it (about 99% smaller). A second function logged one line per item inside a loop over hundreds of thousands of items; info-level logging there was simply removed, keeping errors only.

The maths of a single log line, worked in the report: a Lambda invoked 50 million times a month that logs a 20 KB event on every call writes about 950 GB a month. At around $0.50 per GB ingested that is roughly $475 a month before storage. A 200-byte summary line writes about 9.5 GB, roughly $5 a month, the same 99% reduction the team saw.

3.4 Fix four: stopped paying tolls to talk to itself

Most edge requests were not users at all but the Next.js server-side renderer calling the company's own API through the public CDN and WAF, then back into the same AWS account. Two obstacles blocked a simple fix: the frontend and backend VPCs used the same CIDR range (so peering was impossible), and the CDN was absorbing about three-quarters of those calls as cache hits.

The team first added a Valkey cache next to the frontend and ran it while the CDN was still in path; it reached a 78.8% hit rate versus the CDN's 77.3%. Only then did they switch backend calls onto AWS PrivateLink, which works across overlapping address ranges.

3.5 Results and pattern

The thinking pattern. Nothing here was a bug. Backups, logs and routing were all doing exactly what they had been configured to do. Cost work at this stage is about re-reading old decisions (compliance-driven retention, default log destinations, debug logging, convenient network paths) and asking whether they still match what the business wants. The team also watched slopes, not just levels: a line item that grows a little every day is a lead worth chasing even when it is small.

Lessons to steal.

• Compliance usually requires retention that is defined and followed, not retention that is maximal. Grandfather-father-son rotation keeps recovery points while cutting copies.
• Anything priced per request (CDN, WAF, NAT, API Gateway) punishes internal traffic that hairpins through public edges.
• Shadow-run a replacement cache while the old path is still live, so an under-performer fails harmlessly.
• An LLM with read-only credentials is an effective cost-forensics assistant when asked precise, answerable questions.
• Cost Explorer's latest days are incomplete. Validate with a "control" line item that should not have moved, and with real-time signals such as logs.

4. CS-03 Segment (2016-2017): the million-dollar engineering problem

Field	Detail
Business	Customer-data infrastructure (now Twilio Segment)
Load / team	Tens of thousands of events per second; two engineers, ~3 months
Published	March 2017
Result	Over $1M a year removed from the AWS bill

Most SaaS products get economies of scale early, because one server can serve many users. Segment's costs grew linearly with adoption instead: every customer event had to be processed. By the end of Q3 2016 the AWS bill made up two-thirds of cost of goods sold and was rising faster than usage.

• Made the bill queryable. They enabled AWS detailed billing, which drops raw line items into S3, and loaded them into Redshift. Line-by-line review surfaced about fifteen problem areas that together made up roughly 40% of the monthly bill.
• Cleared incident leftovers. Hundreds of large EBS volumes and over-provisioned cache and RDS instances had been scaled up during past incidents and never scaled back down.
• Fixed DynamoDB hot shards (~$300K a year). Provisioned throughput had been raised table-wide to satisfy a handful of very hot partition keys. Fixing the access pattern removed the need to pay for capacity the rest of the table never used.
• Service auto-scaling (~$60K a year). Services that ran at fixed capacity were put on auto-scaling.
• Bin-packing and fewer instance types (~$240K a year). Consolidating onto fewer, better-packed instance types raised utilisation.

The thinking pattern. The authors were blunt that no tool is a magic bullet; the work is digging through the bill line by line. They also framed cost as a business constraint (gross margin), not a housekeeping chore, which justified dedicating engineers to it. Lessons: put billing data in a warehouse (today: CUR 2.0 or Data Exports queried with Athena or your own warehouse); every incident scale-up should create a follow-up ticket to scale back down; for DynamoDB, capacity is set by the hottest key, so fix key design before buying throughput; a smaller menu of instance types packs better and is easier to commit to.

5. CS-04 Duolingo (2024): cost optimisation and code health go hand in hand

Field	Detail
Business	Language-learning app, 100+ microservices
Trigger	New AI-heavy features (Stories, DuoRadio, Adventures) drove spend into millions a year
Effort	Dozens of engineers, a few months, supported by a five-person FinOps team
Result	20% annualised cloud savings; cleaner codebase

• Visibility first. A cost platform (CloudZero) broke spend into queryable items. One early surprise: a service's staging environment cost more than production because someone had scaled it up for a test and never scaled it back. Cost data was wired into the normal metrics stack and weekly per-team reports, and later extended to non-AWS spend such as LLM APIs.
• Deleted what was not needed. Old ElastiCache clusters, whole databases and an entire microservice belonging to retired features were removed.
• Deleted data, not just resources. S3 buckets in active use were paying to keep their complete version history, so lifecycle rules were added. DynamoDB tables held stale rows because TTLs were missing. Production logging was verbose and unsampled; stack traces in particular were expensive.
• Right-sized memory and task counts. Testing showed nearly all services could run at 90-95% memory utilisation. Minimum task counts were lowered and scale-out made less aggressive; one service alone saved hundreds of thousands of dollars a year.
• Used built-in pricing options. Switching one database to Aurora I/O-Optimized saved several hundred thousand dollars a year; DynamoDB Standard-IA table class was used where it fit; RI purchases were sized to the measured baseline.
• Reduced traffic. Refactoring a legacy service that triggered side effects eliminated about 2.1 billion unnecessary internal calls a day. Raising a cache TTL from one minute to one hour for a rarely changing resource cut traffic to another service by over 60%.

The thinking pattern. Make cost a first-class engineering signal. When owners can see the literal price of their tech debt, they make different choices. Duolingo pushed cost into design documents (cost estimates in tech specs) and into regular team rituals (quarterly spend reviews), and automated anomaly detection across services rather than relying on manual monitoring.

Lessons to steal. Look for staging or test environments that cost more than production. Adding a DynamoDB TTL does not clean up existing rows: backfill the TTL attribute or run a one-off purge. Stack traces and unsampled debug logs are some of the most expensive bytes you will ever write. Microservice fan-out multiplies load-balancer, cross-AZ and task costs, so fix chatty call paths and cache stable data longer. Aurora I/O-Optimized can be dramatically cheaper for I/O-heavy clusters (AWS suggests evaluating it when I/O exceeds about a quarter of Aurora spend).

6. CS-05 Lyft (2018-2019): unit economics as AWS cost per ride

Lyft had committed to at least $300M on AWS for 2019-2021 (per its IPO filing) and grown from about 50M rides in 2015 to over 350M. Hyper-growth had outpaced cost management. The absolute bill was expected to grow with rides, so the useful question was whether each ride was getting cheaper to serve.

• Version 1: a spreadsheet and a script. A Python script divided AWS spend by the number of rides and ranked engineering teams by their spend trend. The ranking had a gamification effect: teams competed to improve.
• Version 2: a proper data pipeline. Because discounts and Reserved Instance costs were not always reflected transparently in raw usage reports, Lyft built a pipeline into a data warehouse with a reporting and dashboard layer queryable in SQL.
• Targeted engineering. The finer-grained data led to halving read/write spend on the top 25 DynamoDB tables and to roughly 50% savings tied to Kubernetes and container migration work.
• Spot for CI. In a separate AWS case study, Lyft cut continuous-delivery infrastructure costs by up to 75% by moving it to Spot Instances, reportedly with a very small configuration change.

The thinking pattern. Normalise cost by the business driver. A rising bill can be healthy; a rising cost per unit never is. Then make that number visible per team so ownership is obvious and peer pressure does some of the work. Lessons: pick one unit metric (cost per ride, order, active user, GB ingested, API call) and chart it weekly; allocate discounts (RIs, Savings Plans, credits) correctly or teams will optimise against misleading numbers; CI/CD fleets are among the safest places to start with Spot.

7. CS-06 Airbnb: treat billing as a data product

At Airbnb's scale, the challenge is less about finding single idle servers and more about making thousands of engineers aware of what their services cost. (Details here come from AWS's published case study as summarised by third parties, so treat specifics as directional; dollar figures were not published.)

• Own the cost data. Airbnb uses Cost Explorer for quick views but loads the Cost and Usage Report into its internal data infrastructure, so spend can be sliced hourly, daily or monthly and attributed to the teams that own it.
• Savings Plans instead of instance-by-instance RIs. Moving EC2 commitments to Savings Plans reduced the operational effort of managing commitments while still producing meaningful savings.
• Intelligent-Tiering for uncertain access. S3 Intelligent-Tiering is used where access patterns are hard to predict.

The thinking pattern. Cost attribution is a data engineering problem. Once billing data sits next to business data, the same analytics stack that powers the product can power cost decisions. Lessons: build on CUR or Data Exports early because a warehouse beats any console for attribution; prefer Savings Plans for flexibility when your instance mix changes often; Intelligent-Tiering charges a small per-object monitoring fee and does not auto-tier objects under 128 KB, so it suits buckets of larger objects with unpredictable access.

8. What the six cases share

Case	How they saw	What they found	Size of win
Levels.fyi 1	Cost Explorer, then inspect inside resources	Idle instances, an orphan cache, stale scaling, old hardware	~15% of the bill
Levels.fyi 2	Line items plus LLM-assisted forensics	Long backup retention, premium log destinations, hairpin traffic	$28.9K a year
Segment	Detailed billing in Redshift, SQL	Incident leftovers, DynamoDB hot keys, fixed capacity	Over $1M a year
Duolingo	Cost platform, weekly reports, anomaly detection	Big staging, unused services, full S3 versions, verbose logs	20% annualised
Lyft	Cost per ride, team rankings, pipeline	Hot DynamoDB tables, container migration, CI	-40% cost per ride
Airbnb	CUR in an internal warehouse	Attribution and commitment overhead	Not published

Three habits recur. First, every team ranked findings by size and by slope, and investigated both lists. Second, they distrusted surface metrics and verified from inside the resource. Third, they left behind a mechanism (a warehouse, a leaderboard, a design-doc section, an anomaly alert) so the savings would not erode.

9. How to start this week

The following is a minimal, copyable version of what these teams did, sized for a single engineer and a few days.

9.1 Make the data queryable

Enable CUR 2.0 or Data Exports to an S3 bucket and create an Athena table over it. A first useful query ranks services by the last full month's unblended cost. The column names below are those of the standard Cost and Usage Report schema; confirm them against your own table, because export options change what exists.

SELECT line_item_product_code AS service,
       ROUND(SUM(line_item_unblended_cost), 2) AS cost
FROM   cur_table
WHERE  line_item_usage_start_date >= DATE '2026-08-01'
  AND  line_item_usage_start_date <  DATE '2026-09-01'
GROUP  BY 1
ORDER  BY cost DESC
LIMIT  20;

Then run the same query per day for the top five services and look at the slope. A line that rises every day, even slowly, is the pattern-P02 lead that led Levels.fyi to a single logging statement.

9.2 Tag before you attribute

Agree a tag schema (owner, service, environment, cost-centre), enforce it through tag policies and infrastructure-as-code checks, and give every temporary resource an expiry tag. The Levels.fyi lesson about "temporary" instances is really a tagging lesson: anything without an owner and an expiry becomes permanent spend.

9.3 Verify from the inside

Before deleting a resource that looks idle, inspect it from within. For a cache that means connected clients and command counters; for a database, active connections and last-write times; for a bucket, access logs or S3 Storage Lens. Take a snapshot or final backup first. If you are unsure, block access for a week and see who complains, which is cheaper than restoring.

9.4 Publish one unit number

Choose the metric, compute it weekly from the same warehouse, and put it in front of the teams. Lyft's first version was a script and a spreadsheet; you do not need a platform to begin.

Should I buy a cost-management platform first?

Duolingo used one and Lyft built pipelines, but the report is explicit that tools matter less than coverage: the wins came from getting many engineers to look at cost data they could act on. Start with CUR or Data Exports and SQL, then add a platform if you outgrow it.

How do I know a "quiet" resource is really unused?

Do not trust activity graphs. Levels.fyi found an ElastiCache cluster that referenced no service yet showed CPU and network activity; the activity was its own backup job. Connect and inspect the internals (clients, command counts), snapshot, then delete.

Can we drop retention to save on backups without failing an audit?

Usually yes. Compliance frameworks such as SOC 2 ask for a defined retention period that you actually follow, not a maximal one. Levels.fyi moved to a tiered policy and cut backup storage from 15.7 TB to 3 TB. Document the policy and confirm the wording with your auditor.

Takeaways

• Start from the bill. All six teams made billing data queryable before touching architecture.
• Almost nothing was a bug. Old decisions (retention, log destinations, incident scale-ups, autoscaling rules) were correct once and expensive later.
• Watch slopes. A $26 line item that grows daily led to a whole clean-up.
• Verify from the inside before deleting, and snapshot first.
• Per-request services punish hairpins. Internal traffic should not pay CDN, WAF or NAT prices.
• Normalise by a business unit and leave a mechanism behind: a warehouse, a leaderboard, a design-doc section, anomaly alerts.

References & further reading

• Cutting the AWS Bill: 28 Case Studies (PDF) - chapter A (CS-01 to CS-06) is the basis of this part.
• Levels.fyi - Slashing our AWS bill (Part 1) · Part 2
• Segment - The million dollar engineering problem
• Duolingo - Reducing cloud spending · Duolingo FinOps
• TechTarget - re:Invent 2019 coverage (Lyft, HERE, Salesforce)
• AWS - Airbnb optimizes usage and costs