Commitments, architecture and network topology: where the invisible costs hide.

Two very different levers share this part because both are about where the money hides once the obvious waste is gone. Commitments (HERE Technologies) need no engineering, which is exactly why they belong last: HERE moved close to 80% of EC2 usage under Reserved Instances and saved about $50M plus $150M avoided over roughly 3.5 years, but a commitment discounts whatever footprint you have, waste included. Architecture, code and network topology (Amazon Prime Video, Recall.ai, DoorDash, nOps) hide in costs that look like ordinary usage: Step Functions per-transition billing, WebSocket memory copies at 93 MB/s per stream, cross-AZ fan-out and a single NAT route worth $12K a month. This part walks through each case, the arithmetic behind it, and the 2025-2026 changes (Database Savings Plans, zone-aware routing in ECS Service Connect) that alter the playbook.

1. Two levers that reward careful thinking

Most of what you have read so far changes what you consume. This part covers two levers that change either how you pay (commitments) or how your design consumes (architecture, code and topology). Both can produce large numbers, and both have a common failure mode: doing them in the wrong order or without measuring.

Commitments are the one lever that requires no engineering, which is exactly why they should come last: they discount whatever footprint you have, including its waste. Architecture and network costs are the opposite: invisible to billing dashboards because they look like ordinary usage, and only found by profiling code, tracing bytes and estimating cost per unit of work for each design.

2. CS-18 HERE Technologies: "let the builders build", then commit centrally

Field	Detail
Business	Mapping and location data
Estate	500+ AWS accounts; 10M+ EC2 instances launched per year
Lever	Programmatic Reserved Instance coverage
Presented	AWS re:Invent 2019
Result	$50M saved plus $150M avoided over ~3.5 years

HERE deliberately avoided procurement gates that slow engineers down. The result was a sprawling estate that needed a different kind of cost control. Rather than restricting teams, HERE adopted Reserved Instances systematically, moving close to 80% of EC2 usage under RIs at roughly 50% off On-Demand rates, and kept builders unblocked. Cost control came from central pricing optimisation rather than from rigid sourcing processes for teams.

The thinking pattern. Separate the question "what should we run?" (owned by builders) from "how should we pay for it?" (owned centrally). Commitment management is a portfolio problem: cover the stable baseline, and leave the variable top On-Demand or Spot.

Lessons to steal.

• Right-size and remove waste first, then commit to the new, lower baseline.
• Today, Compute Savings Plans offer similar discounts with flexibility across instance families, regions, Fargate and Lambda.
• Database Savings Plans (announced December 2025) bring the same model to RDS, Aurora, DynamoDB, ElastiCache (Valkey) and more: up to 35% off with a one-year, no-upfront commitment.
• Track two KPIs: coverage (share of eligible usage discounted) and utilisation (share of commitment actually used).

3. Sizing a commitment

The report's section on maths gives a compact procedure that applies to Savings Plans, Reserved Instances and Database Savings Plans alike.

1. Finish the cheaper work first. Right-sizing, Graviton moves and scheduling all change the baseline; a commitment freezes it.
2. Commit to the trough, not the average. Look at hourly eligible spend over the last 30-60 days and commit to roughly its minimum sustained level. Anything above that stays On-Demand or Spot.
3. Ladder the purchases. Buy quarterly, for example, so commitments expire at different times and can track change.
4. Track coverage and utilisation. Aim high on utilisation before pushing coverage; an unused commitment is a pure loss.
5. Prefer the flexible instrument (Compute Savings Plans, Database Savings Plans) while your instance families or engines are still changing.

KPI	Definition	Healthy target
Commitment utilisation	Used commitment / purchased commitment	Above 95%
Commitment coverage	Discounted eligible usage / eligible usage	70-85% of the steady baseline

3.1 A worked illustration

Suppose eligible compute spend, measured hourly over 60 days, has a maximum of $90 an hour, an average of $70 and a sustained minimum of $50. Committing to the average ($70 an hour) means that during the quiet hours you pay for $20 an hour you do not use; over a year that is roughly $175,000 of committed spend that earns nothing. Committing to the trough ($50 an hour) leaves the variable top On-Demand or Spot but wastes nothing. Because the discount applies only to what you cover, the saving is smaller per hour but certain. Laddering lets you add a second commitment when the trough rises. The numbers are illustrative, not from a case, but the shape is the one HERE used: cover the stable baseline, leave the variable top flexible.

4. What changed in 2025-2026

• Database Savings Plans (December 2025). One-year, no-upfront commitments of dollars per hour that discount Aurora, RDS, DynamoDB, ElastiCache (Valkey), DocumentDB, Neptune, Keyspaces, Timestream and DMS by up to 35%, across engines, instance families and regions. Serverless options such as Aurora Serverless v2 and DynamoDB on-demand became discountable for the first time. Discounts vary by service: RDS and Aurora instances about 20%, their serverless forms up to 35%.
• Resale and pooling rules (2025). AWS reportedly restricted Reserved Instances and Savings Plans from June 2025 to a single end customer's usage, limiting pooled "group buying" across unrelated customers. Re-check any third-party commitment service you rely on (Levels.fyi's group-buying saving of about $700 a month is an example of what is now affected).
• Zone-aware routing in ECS Service Connect (July 2026). On by default; keeps most service-to-service traffic in-zone, cutting cross-AZ charges and latency without code changes (section 8).

5. CS-19 Amazon Prime Video: when serverless orchestration costs more than the work

Field	Detail
Business	Video streaming (inside Amazon)
Service	Audio/video quality monitoring of live streams
Change	Step Functions + Lambda + S3 to a single-process ECS service
Published	2023
Result	~90% lower infrastructure cost for this service

The Video Quality Analysis team built a tool to inspect every stream for defects such as corruption or audio/video sync problems. The first version used distributed serverless components orchestrated by Step Functions, with S3 as the hand-off point for video frames between components.

• Found the per-transition trap. The workflow performed several state transitions for every second of every stream. That hit account limits, and because Step Functions bills per state transition, cost grew with stream-seconds.
• Found the per-request trap. Passing frames between components through S3 generated an enormous number of S3 requests.
• Collapsed to one process. All components (media conversion, detectors, orchestration) were packed into a single process running on ECS/EC2, with frames passed in memory. To scale beyond one instance, the service is cloned and each clone runs a subset of detectors, with a lightweight layer distributing requests.

The thinking pattern. Granularity is a cost decision. Serverless and microservices are priced per invocation, transition or request; that is excellent for spiky, low-volume work and ruinous for high-frequency inner loops. The team moved the inner loop into memory and kept distribution only at the coarse level where it paid off.

Lessons to steal.

• Estimate cost per unit of work (per stream-second, per event) for each design before building it.
• Never use object storage or a queue as the data bus for a tight inner loop.
• For high-volume, short workflows, Step Functions Express workflows are billed by requests and duration rather than per transition.
• This is not an argument against serverless in general; it is an argument for choosing granularity per workload.

5.1 Cost per unit of work, applied

The practical tool from this case is a small table you can build in a design review. For each candidate design, count the billable events per unit of work, multiply by the unit price, and compare. Illustratively, if a pipeline performs four state transitions per second per stream and there are 10,000 concurrent streams, that is 40,000 transitions per second, or over 100 billion a month, before any Lambda or S3 charges; the arithmetic is what makes the design visibly non-viable, not a specific price. The same table forces you to write down what each hop costs in requests, in bytes and in transitions. Do it before building, and the trap Prime Video fell into becomes obvious on paper.

6. CS-20 Recall.ai: how WebSockets over loopback cost $1M a year

Field	Detail
Business	Meeting-bot API (Zoom, Meet, Teams)
Load	Millions of meetings per month; ~1 TB/s of raw video IPC
Fleet	CPU-based video processing on AWS
Published	October 2024
Result	Over $1M a year saved by replacing an IPC mechanism

Each bot ran a headless Chromium to join meetings plus a real-time media pipeline, and needed about four CPU cores. The team set a goal to halve that, which would roughly halve the compute bill.

• Profiled before guessing. CPU profiles showed a surprising hot spot: memory-copy routines, called mostly by the Python WebSocket client receiving data and by Chromium's WebSocket implementation sending it.
• Understood the data volume. Raw decoded video was being streamed out of Chromium's JavaScript environment over a local WebSocket. Uncompressed 1080p at 30 fps is roughly 93 MB/s per stream (1920 x 1080 x 1.5 bytes per frame), so every extra copy was expensive.
• Built a better pipe. After evaluating alternatives such as TCP and Unix domain sockets, the team built a shared-memory ring buffer (including modifying Chromium) so frames could move between processes without repeated copying and framing.

The thinking pattern. In a CPU-bound fleet, every percentage point of CPU is a percentage point of the bill. Profiling found a cost that no billing dashboard could have revealed, because it looked like ordinary EC2 usage.

Lessons to steal. For compute-dominated bills, a profiler (perf, flame graphs, py-spy) is a cost tool. Estimate bytes per second through each hop of a pipeline; high-bandwidth IPC needs zero-copy designs. Set a concrete efficiency goal (for example "half the cores per unit of work") and work backwards.

The arithmetic that makes the case clear is worth doing once: 1920 x 1080 pixels at 1.5 bytes per pixel (YUV 4:2:0) is about 3.1 MB per frame, and at 30 frames a second that is roughly 93 MB/s, per stream. Multiply by many concurrent streams and every unnecessary copy (into a socket buffer, out into user space, through framing) is a large multiple of a very large number. Profiling turned that from a vague suspicion into a named function.

7. CS-21 DoorDash: staying in the zone

Field	Detail
Business	Food delivery marketplace
Architecture	Cells + microservices on Kubernetes, Envoy service mesh
Change	Zone-aware routing in the mesh
Published	January 2024
Result	Cross-AZ traffic localised inside each Availability Zone

Moving from a monolith to cells and microservices improved development speed and resilience, but cross-AZ data transfer costs rose as the number of service-to-service hops grew. On AWS, traffic between AZs is billed in both directions.

• Classified the traffic. Traffic was split into service-to-service HTTP/gRPC, storage traffic (Aurora PostgreSQL, CockroachDB, Redis, Kafka) and internal infrastructure traffic such as DNS and control-plane chatter.
• Tested a hypothesis. The team suspected HTTP/gRPC calls within a cell were the largest source and prioritised them.
• Enabled zone-aware routing. Their in-house service-mesh control plane was extended to give Envoy each endpoint's AZ, so callers prefer same-zone targets and spill over only when a zone lacks healthy capacity. The cell-based design further localised traffic.

The thinking pattern. Resilience and cost are both properties of topology. Keep traffic local by default, and cross zones only when needed.

Lessons to steal. Cross-AZ transfer is charged on send and receive (about $0.01 per GB each way in most regions), and microservice fan-out multiplies it. Balance capacity per AZ, because zone-aware routing on an unbalanced fleet overloads the small zone. New in July 2026, Amazon ECS Service Connect has zone-aware routing on by default, keeping over 80% of traffic local when endpoints are balanced; Kubernetes offers topology-aware routing for Services.

7.1 Cross-AZ chatter, worked

The report gives a clean example. Two services exchanging 50 TB a month across AZs pay about $0.01 per GB in each direction, around $1,000 a month. If zone-aware routing keeps 80% of that traffic local, the charge falls to roughly $200. The saving grows linearly with every new hop in a microservice call chain, which is why a five-hop request path is five times the exposure of a single call.

8. CS-22 nOps (its own AWS account): $12K a month hiding in one NAT route

Field	Detail
Business	Cloud cost-management vendor
Symptom	Bandwidth charges up by thousands with no clear cause
Tools	VPC Flow Logs plus cost analytics
Published	2025
Result	~$12,000 a month removed by fixing a single routing issue

Even a FinOps company can be surprised by NAT gateway charges, which are billed per hour and per GB processed and are easy to miss in coarse cost views. VPC Flow Logs showed which sources and destinations were pushing traffic through the NAT gateway. A misconfiguration was sending traffic through NAT that did not need to go there; correcting one route removed the charge.

The thinking pattern. Data-transfer costs are invisible until you attribute bytes to flows. Flow logs turn a mysterious network line item into a list of talkers.

Lessons to steal. Add S3 and DynamoDB gateway endpoints everywhere (they are free) and check that every private route table actually has the prefix-list route. Add interface endpoints for ECR, CloudWatch, STS and other chatty AWS APIs when volume justifies them. Alert on NatGateway-Bytes growth that does not match workload growth.

8.1 NAT gateway versus endpoint, worked

A workload in private subnets pushing 10 TB a month to S3 through a NAT gateway pays about 10,240 GB times $0.045, roughly $461 a month in processing, plus about $33 per NAT gateway per month in hourly charges (times the number of AZs). Through an S3 gateway endpoint the same traffic costs nothing extra. For ECR image pulls, CloudWatch or STS, interface endpoints cost about $0.01 per hour per AZ plus $0.01 per GB, which beats NAT once volume is more than a few hundred GB a month.

Path	Cost driver	Best used for
NAT gateway	~$0.045 per GB processed plus hourly fee per gateway	General internet egress that has no alternative
S3 / DynamoDB gateway endpoint	Free	All S3 and DynamoDB traffic from private subnets
Interface endpoint	~$0.01 per hour per AZ plus ~$0.01 per GB	Chatty AWS APIs (ECR, CloudWatch, STS) at volume
Public IPv4 address	~$0.005 per hour each	Avoid; prefer private subnets and endpoints

9. Finding these costs in your own estate

The four architecture and network cases share one method, and it is different from the billing-first method of Part 2 because the problem is in bytes and requests, not dollars. A reusable sequence:

1. Get the top network line items. In CUR, filter on usage types containing DataTransfer, NatGateway and Regional-Bytes; rank them.
2. Turn on VPC Flow Logs, sampled. Flow logs have their own cost, so sample or scope them to the subnets in question. Aggregate by source, destination and port to build a list of talkers.
3. Classify traffic the way DoorDash did: service-to-service HTTP/gRPC, storage replication, and infrastructure chatter. Prioritise the class your hypothesis says is largest, then test the hypothesis.
4. Profile compute-heavy services. If the bill is CPU-bound, run a profiler and set a "cores per unit of work" target (Recall.ai).
5. Estimate cost per unit of work for any per-request or per-transition design and compare against an in-process alternative (Prime Video).

10. Putting commitments in the right place

HERE's approach is instructive precisely because it succeeded at scale: a central team owned pricing, builders stayed unblocked, and coverage was built up programmatically. The consistent order across the whole series is optimise, then commit. If you fix the NAT route, collapse a hot loop and halve CPU per unit of work (as Recall.ai did) before buying, the commitment is smaller, cheaper and safer. If you buy first, the commitment is sized to a footprint you are about to shrink, and the discounts strand.

Should we buy Savings Plans or Reserved Instances?

Prefer Compute Savings Plans and Database Savings Plans while your families or engines are still changing, since they are flexible across families, regions, Fargate and Lambda. Reserved Instances can be slightly less flexible; HERE used RIs at roughly 50% off. Either way, commit to the trough of the optimised baseline.

Did Prime Video prove that microservices are wasteful?

No. The report is explicit that the lesson is narrower: match the pricing model to the traffic shape. Serverless is excellent for spiky, low-volume work and ruinous for high-frequency inner loops billed per transition or request.

How do I find a NAT problem like nOps's?

Look at NAT gateway bytes in CUR or CloudWatch, then use VPC Flow Logs to list which sources and destinations produce the traffic. Add S3 and DynamoDB gateway endpoints, verify the route tables carry the prefix-list route, and alert when NAT bytes grow faster than the workload.

Takeaways

• Commit last, and to the trough. Commitments discount waste; size them to the optimised baseline and ladder purchases.
• Separate "what to run" from "how to pay". Builders own the first, a central team the second (HERE).
• Granularity is a cost decision. Per-request and per-transition services punish tight inner loops (Prime Video).
• Profile when compute dominates. A memory copy in a WebSocket cost $1M a year (Recall.ai).
• Keep traffic in-zone by default (DoorDash) and route AWS API traffic through endpoints, not NAT (nOps).
• Attribute bytes, not just dollars. Flow logs turn a network line item into a list of talkers.

References & further reading

• Cutting the AWS Bill: 28 Case Studies (PDF) - chapters E and F (CS-18 to CS-22) and sections 6.4-6.6, 6.8.
• TechTarget - re:Invent 2019 coverage (HERE)
• AWS - Database Savings Plans announcement
• DevClass - Prime Video case study coverage
• Recall.ai - How WebSockets cost us $1M on our AWS bill
• DoorDash - Staying in the zone · AWS - zone-aware routing in ECS Service Connect
• nOps - reduce NAT gateway costs · Vantage - NAT gateway vs endpoints