Elasticity that pays: Spot, Karpenter and switching things off.

Elasticity is the cloud's core promise, but it only pays off when workloads can be moved, interrupted or switched off. Four case studies show what that discipline looks like: Salesforce Audience Studio runs more than 85% of a platform on Spot for 48% savings, Tinybird moved from Cluster Autoscaler to Karpenter with consolidation and cut its AWS bill about 20% (up to 90% for CI/CD), Salesforce's huge EKS estate gained roughly 5% in the first year with 5-10% more expected, and Moneysmart combined mixed Spot pools with a "non-working hours" shutdown policy. The unifying lesson is that autoscalers can only save money that workloads allow them to save, so reliability engineering and cost engineering become the same work. This part gives you the Spot and Karpenter checklists, the schedule maths (a week has 168 hours, people work in about 50 of them) and a safe order of adoption.

1. Elasticity is a property of the workload, not the autoscaler

Every provider sells elasticity, and every finance team assumes it is happening. In practice, savings only appear when workloads tolerate being moved, interrupted or stopped. Spot capacity can be reclaimed on short notice; Kubernetes consolidation evicts pods routinely; a scheduled shutdown deletes running work every evening. If a service cannot handle any of those events gracefully, the autoscaler simply cannot save money, and forcing it will cause outages instead.

This is why the four cases in this part read more like reliability stories than pricing stories. Salesforce Audience Studio designed for interruption from the start. Tinybird hardened its pods for movement before turning consolidation on. Moneysmart provided a way for engineers to wake environments back up. Each team paid the workload-discipline cost first and collected the savings second.

2. CS-14 Salesforce Audience Studio: 85% of a big-data platform on Spot

Field	Detail
Business	Data-management platform (from the Krux acquisition)
Load	~200,000 user queries per second; ~2,500 EMR clusters
Presented	AWS re:Invent 2019
Result	48% overall savings after moving to Spot

Audience Studio processes large volumes of third-party audience data in parallel batches, the ideal shape for Spot capacity. More than 85% of the platform's infrastructure runs on Spot Instances, which are spare EC2 capacity sold at a steep discount but reclaimable at short notice. Jobs are parallel, batch-oriented and restartable, so losing an instance means redoing a slice of work rather than failing a customer request. The result was 48% overall savings across roughly 2,500 daily EMR jobs.

The thinking pattern. Spot is an architecture decision, not a purchasing decision. If a workload is parallel, restartable and flexible about instance type, Spot should be the default and On-Demand the exception.

Lessons to steal.

• Diversify across many instance types and Availability Zones, and use capacity-optimised allocation to reduce interruptions.
• Checkpoint long tasks so an interruption costs minutes, not hours.
• Keep the driver or master nodes On-Demand and run executors on Spot.

3. CS-15 Tinybird: EKS + Karpenter + Spot, and the discipline consolidation demands

Field	Detail
Business	Real-time analytics platform
Change	Cluster Autoscaler node groups to Karpenter, on production and CI/CD in EKS
Published	June 2025
Result	~20% off the total AWS bill; up to 90% for CI/CD workloads

3.1 Context

With Cluster Autoscaler, every combination of instance type, zone and purchase option needed its own node group, so the team was micromanaging node groups and still wasting capacity.

3.2 What they did

• Adopted Karpenter. Karpenter launches right-sized nodes directly from pending pods' requirements, choosing from many instance types, including Spot.
• Turned on consolidation. Karpenter continually looks for under-used nodes, moves their pods, and removes or replaces the nodes with cheaper ones, even when pod counts do not change.
• Hardened workloads for movement. Because consolidation evicts pods routinely, services got proper graceful-shutdown handling and related safeguards so that moves do not cause errors or micro-outages.

The thinking pattern. Autoscalers can only save money that workloads allow them to save. Consolidation and Spot both require pods that can be moved at any time, so reliability engineering and cost engineering become the same work.

Lessons to steal. Before enabling consolidation, set terminationGracePeriodSeconds, handle SIGTERM, and add readiness probes and PodDisruptionBudgets. Right-size pod requests first, because Karpenter packs requests, not actual usage. CI runners are the safest place to start with Spot on Kubernetes.

3.3 A Karpenter configuration sketch

The report describes Tinybird's approach in prose; the following is an illustrative sketch of the Karpenter concepts it names (flexible instance requirements, Spot alongside On-Demand, and consolidation), not Tinybird's actual configuration. Field names follow the Karpenter v1 API; verify against the version you run.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64", "amd64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m
    budgets:
      - nodes: "10%"

Two ideas in that sketch carry the most weight. The wide instance-category and architecture requirements let Karpenter pick the cheapest fit (this is the "diversify across many types" lesson from Audience Studio, applied to Kubernetes). The disruption budget caps how many nodes consolidation may touch at once, which is what keeps churn from becoming an outage.

3.4 The workload side: what "hardened for movement" means

None of the tooling matters if pods die badly. The minimum a stateless service needs before you enable Spot or consolidation:

spec:
  terminationGracePeriodSeconds: 60
  containers:
    - name: service
      lifecycle:
        preStop:
          exec:
            command: ["sh", "-c", "sleep 10"]   # let the endpoint drain
      readinessProbe:
        httpGet: { path: /ready, port: 8080 }
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: service
spec:
  minAvailable: 80%
  selector:
    matchLabels: { app: service }

The application must also handle SIGTERM by finishing in-flight requests and refusing new ones. Without that, the first consolidation wave produces a burst of failed requests, which is exactly the churn the watch-out warns about. Stateful or slow-starting pods should carry a karpenter.sh/do-not-disrupt annotation until you have engineered them for movement.

4. CS-16 Salesforce (EKS fleet to Karpenter): single-digit percentages at enormous scale

Field	Detail
Business	Enterprise SaaS
Change	Cluster Autoscaler to Karpenter across a very large EKS estate
Primary driver	Faster onboarding, less platform-team toil
Reported	InfoQ, January 2026: ~5% savings in FY2026, with a further 5-10% expected in FY2027

At Salesforce's scale the migration was motivated mainly by operational agility: faster onboarding and less dependence on central platform teams. Karpenter's real-time instance selection (including Arm, GPU and Spot) and bin-packing replaced Cluster Autoscaler. Cost was a co-benefit of a platform modernisation: about 5% in the first year, with another 5-10% anticipated as bin-packing and Spot usage mature.

The thinking pattern. For very large fleets, a few percent is a large absolute number, and the gains compound as teams adopt the new capabilities. Lessons: frame platform migrations with both an operational and a cost business case; expect savings to arrive in stages as workloads adopt Spot and tighter requests. Watch-out: absolute dollar figures were not disclosed, so use this case for direction, not magnitude.

5. CS-17 Moneysmart Group: Spot pools and a "non-working hours" policy

Field	Detail
Business	Personal-finance comparison (Southeast Asia)
Platform	ECS and EKS
Tools	Cost Explorer, Trusted Advisor, Spot.io, Lambda
Result	~58% net saving on Spot-managed capacity after vendor fees

A growing startup analysed its bill with Cost Explorer and Trusted Advisor and worked with its AWS account team to pick focus areas.

• Mixed Spot pools. A Spot management service ran mixed Spot pools for ECS and EKS with container-driven autoscaling and right-sizing. In one example month they paid about $2,078 for Spot usage while saving about $5,656; the vendor took 20% of the savings, leaving roughly 58% net.
• Switched non-production off at night. A Lambda function scaled staging EC2 down from 22:00 to 07:00 on weekdays and for the whole weekend.

The thinking pattern. A week has 168 hours; people work in roughly 50 of them. Non-production that runs around the clock pays for more than two-thirds idle time.

Lessons to steal. Running staging only 07:00-22:00 on weekdays (75 of 168 hours) cuts its compute cost by about 55%. Compare managed-Spot vendor fees (often a share of savings) with the cost of running Karpenter or Spot Fleet yourself. Watch-out: scheduled shutdowns need a self-service "wake up" path, or engineers will simply disable them.

6. The schedule maths

The following table is straight from the report and is worth memorising. It shows the compute saving of switching a non-production environment off outside working hours, relative to running it 24 hours a day, seven days a week.

Schedule	Hours per week	Compute saving vs 24x7
24 x 7	168	0%
Weekdays 07:00-22:00 (Moneysmart)	75	55%
Weekdays 08:00-20:00	60	64%
Weekdays 09:00-19:00	50	70%

The saving is one minus the hours-on divided by 168. For the Moneysmart schedule, 1 - 75/168 is about 0.55. Two caveats: only compute stops billing when an instance is stopped (attached EBS volumes and Elastic IPs continue to bill), and managed databases may not support the same stop-start pattern without their own automation. Both are reasons to write the schedule as code and to review the resources it does not cover.

6.1 The wake-up path

Moneysmart's lesson generalises: a schedule that engineers cannot override gets disabled by them. The lightest fix is a chat command or a one-click pipeline that starts an environment for a fixed window (say four hours) and lets the schedule take it down again afterwards. Log the wake-ups; if the same environment is woken every night, the schedule is wrong for that team and should move, not be removed. Kubernetes teams can achieve the same with an Argo CD-reconciled scale-to-zero for non-production namespaces (Part 9).

7. Is it a Spot candidate?

Question	What a good answer looks like
Stateless or checkpointed?	Losing the node loses at most a slice of work
Tolerates a two-minute termination notice?	Can drain and hand off within the warning window
Runs on several instance types?	Not tied to one family, so allocation can diversify
Horizontally scaled?	Capacity comes from many small nodes, not one big one
Retries are cheap?	A restarted task is a small cost, not a customer-visible failure

Batch analytics, CI runners, stateless web tiers behind a load balancer and build farms fit well. Primary databases, single-instance leaders and anything with long, uncheckpointed state do not. The safest first target on Kubernetes is CI, which is why both Lyft (Part 2, up to 75% off) and Tinybird (up to 90% off) started there.

8. Managed Spot versus doing it yourself

Moneysmart paid a vendor 20% of the savings and kept roughly 58% net. That trade can be right for a small team without the time to run Spot Fleet or Karpenter. It stops being right as the savings grow, because a percentage of savings scales with your success while your own operating cost does not. The report's suggestion is to compare the vendor fee with the cost of running Karpenter or Spot Fleet yourself, and to revisit the decision as the estate grows. Also re-check any commitment-pooling arrangement: AWS reportedly restricted RIs and Savings Plans to a single end customer's usage from June 2025.

9. A safe order of adoption

1. Schedules on non-production. Highest certainty, near-zero risk to customers, immediate payback (55-70% of that compute).
2. Right-size pod and task requests from real usage (Karpenter packs requests, not usage).
3. Harden workloads for movement: SIGTERM handling, readiness probes, PDBs, checkpointing.
4. Spot for CI and batch, with diversified instance types and capacity-optimised allocation.
5. Consolidation on stateless services, with disruption budgets and do-not-disrupt for the rest.
6. Spot for stateless production tiers, only after observing interruption behaviour on CI and batch.

10. Spot mechanics worth knowing

Three facts drive good Spot design, and all three appear in the cases even where they are not named.

• You get a short warning, not a guarantee. When AWS needs the capacity back it sends an interruption notice about two minutes ahead. Everything your workload needs to do to stay correct (finish or hand off requests, flush state, deregister from the load balancer) must fit in that window. This is why the Audience Studio lesson is to checkpoint long tasks so that an interruption costs minutes rather than hours.
• Diversity lowers the interruption rate. Spot capacity is a set of independent pools, one per instance type and Availability Zone. A fleet that can use ten types across three zones has thirty pools to draw from, so a shortage in any one of them matters less. Capacity-optimised allocation strategies steer toward the pools least likely to be reclaimed.
• The discount varies, and can approach 90%. The report's catalogue quotes up to about 90% off On-Demand, but a realistic blended figure across a diversified fleet is lower, and the managed-Spot example at Moneysmart nets about 58% after the vendor's 20% share of savings.

11. CI runners: the safest place to begin

Both Lyft (up to 75% off its continuous-delivery infrastructure, reportedly with a very small configuration change) and Tinybird (up to 90% off its CI/CD workloads) found their largest, safest saving in CI. The reasons are structural. CI jobs are short and independent, so an interruption costs one job's worth of work. A failed job is retried automatically or by the developer, so an occasional reclaim is annoying, not customer-facing. And CI demand is spiky, which is where Spot plus autoscaling shines.

A workable pattern for self-hosted runners (for example with GitHub Enterprise) is an autoscaling runner set on Spot, ideally arm64 as well, with a hard timeout per job so a stuck run cannot hold a node forever. Set artifact and cache retention, and cache container layers so repeated base-image pulls do not become NAT egress (Part 6). Measure success by cost per build, not just total spend.

12. Right-sizing containers on ECS and Fargate

Fargate tasks are billed on the vCPU and memory you request, not on what they use. Duolingo (Part 2) found that testing showed nearly all its services could run at 90-95% memory utilisation, so it lowered task sizes and minimum task counts and made scale-out less aggressive; one service alone saved hundreds of thousands of dollars a year. The method is portable:

1. Pull a month of memory and CPU utilisation percentiles per service (CloudWatch Container Insights or your metrics stack).
2. Set requests just above the observed high percentile plus a safety margin, and test under load in a lower environment first.
3. Lower the minimum task count where traffic allows, keeping enough headroom for cold starts.
4. Recheck after each release, because request drift quietly undoes the work.

Right-sizing is also the prerequisite for everything else in this part: consolidation packs requests, so inflated requests waste nodes, and Spot savings are capped by the size of what you ask for.

13. A footnote on Kubernetes version support

Elasticity work often touches clusters, and the report's technique catalogue includes one fixed cost that is easy to miss: EKS clusters left on versions outside standard support carry a much higher control-plane fee, about $0.60 per cluster-hour instead of about $0.10. Each cluster also carries a fixed hourly fee regardless of size, so a sprawl of small clusters adds up. Two habits keep this in check: upgrade on a cadence so nothing drifts into extended support, and count clusters, consolidating where isolation is not required. Multi-cloud teams should note that GKE and OKE have equivalent controls (Spot VMs, preemptible instances, flexible shapes), though the price points differ.

14. Elasticity anti-patterns

Anti-pattern	Why it hurts	Fix
Spot or consolidation without graceful shutdown	Routine evictions become error bursts (Tinybird's prerequisite)	SIGTERM handling, readiness probes, PDBs first
Consolidating before right-sizing requests	Karpenter packs requests; inflated requests waste nodes	Use VPA recommendations or observed usage to set requests
A schedule with no wake-up path	Engineers disable it (Moneysmart caveat)	Self-service temporary start with automatic re-stop
One instance type on Spot	A single pool shortage interrupts everything	Diversify types and zones; capacity-optimised allocation
Vendor fee as a percentage forever	The fee grows with your savings	Re-compare against self-managed as the estate grows
Stateful pods in the consolidation pool	Churn, slow restarts, data risk	do-not-disrupt annotation and disruption budgets
Stopping instances but forgetting attached storage	EBS and Elastic IPs still bill	Review what the schedule does not stop

Every row is a case of the opposite of pattern P10: making workloads movable before letting automation move them. The tools are cheap; the discipline is the work.

Is Spot safe for production web services?

It can be, for stateless tiers behind a load balancer with diversified instance types and graceful shutdown, but the cases start with batch and CI. Salesforce runs 85% of a batch platform on Spot; Tinybird's biggest win (up to 90%) is CI/CD. Move stateless production only after you have observed interruption behaviour and hardened the workloads.

Why does Karpenter save less than the headline percentages at large companies?

Salesforce reports about 5% in the first year, with 5-10% more expected. At very large scale a few percent is a large absolute number, and gains arrive in stages as teams adopt tighter requests and Spot. Tinybird's 20% came from a whole bill with a large CI share.

What breaks first when consolidation is enabled?

Pods that do not handle SIGTERM, lack readiness probes, or have no PodDisruptionBudget. Stateful and slow-starting pods suffer from churn. Fix the workload side before switching consolidation on.

Takeaways

• Spot is an architecture decision, not a purchasing decision. Parallel, restartable, instance-flexible work should default to Spot.
• Autoscalers only save what workloads allow. Harden for movement first: SIGTERM, readiness, PDBs, checkpoints.
• Karpenter packs requests, not usage, so right-size requests before consolidating.
• A week has 168 hours and people work in about 50. Scheduling non-production saves 55-70% of its compute.
• Provide a wake-up path, or engineers will switch the schedule off.
• Compare managed-Spot fees with running it yourself, and start with CI, the safest workload.

References & further reading

• Cutting the AWS Bill: 28 Case Studies (PDF) - chapter D (CS-14 to CS-17) and section 6.3 (scheduling table).
• Tinybird - How we cut AWS costs with EKS, Karpenter and Spot
• InfoQ - Salesforce EKS fleet to Karpenter
• AWS Startups Blog - Moneysmart Group
• TechTarget - re:Invent 2019 coverage (Salesforce Audience Studio)
• Karpenter - disruption and consolidation