AWS re:Invent 2025 - Supercharge your Karpenter: Tactics for smarter K8s optimization (COP208)

AWS Events

4 views • 7 months ago Save 7 min 7 min read

Video Summary

This video explores how to achieve cost efficiency and adaptive scaling in Kubernetes environments using Carpenter. It highlights key strategies such as leveraging consolidation policies with PDBs and delays, diversifying instance types, managing DaemonSets effectively, and utilizing spot instances for interruptible workloads with a failover to on-demand. The discussion also touches on using Graviton instances for performance and cost benefits, and the potential cost savings from limiting node pools to specific availability zones for non-production environments. A significant challenge identified is the difficulty in connecting Kubernetes metrics to costs and the operational complexity of continuous optimization.

The video introduces Zesti as a platform that addresses these challenges by providing deep visibility, predictive scaling, and automated resource and financial optimization. Zesti's fast scale technology maintains pre-cached, hibernated nodes ready in 30 seconds, enabling instant recovery from spot instance failures and facilitating the migration of more workloads to spot instances with reduced risk. One interesting fact is that Zesti claims to provide at least 70% cost savings by combining resource and financial optimization practices.

Short Highlights

Carpenter helps achieve cost efficiency and adaptive scaling in Kubernetes by managing underutilized nodes for consolidation and moving containers.
Key strategies include using PDBs to control replica disruption and consolidation delays to prevent interruptions during peak hours.
Diversifying instance types across families, generations, and sizes, while avoiding extremes (too small or too large), optimizes Carpenter's instance selection.
Spot instances, when managed with a failover to on-demand and appropriate node pools, can provide significant cost savings for interruptible workloads.
Zesti offers a platform for continuous automation, providing deep visibility, predictive scaling, and automated resource and financial optimization, claiming up to 70% cost savings.

Key Details

The Challenge of Cost vs. Performance in Kubernetes [0:58]

Throwing money at the problem by using the largest, most powerful instances leads to amazing performance but at a terrible cost.
Scaling down to the bare minimum can result in unavailability, making efficiency a difficult balance to strike.
DevOps and platform engineers aim to find the "sweet spot" by providing the right infrastructure for complex workloads, avoiding over or under-provisioning.
This balance is tough, especially in Kubernetes, where constant configuration tuning is required for efficiency.

"So as DevOps engineers and platform engineers our goal is to find the sweet spot."

Carpenter's Consolidation Policies and Guardrails [2:01]

Carpenter's powerful consolidation policies monitor nodes for underutilization and attempt to safely terminate them to save costs by moving containers elsewhere.
Pod Disruption Budgets (PDBs): This is a crucial guardrail that controls how many replicas can be consolidated simultaneously, ensuring a minimum number of replicas remain available.
Consolidation Delays: Configuring delays prevents Carpenter from terminating nodes during short-term dips in usage, avoiding unnecessary interruptions.
Implementing these controls allows users to avoid consolidation during peak traffic hours.

"Number one is using what's called PDB or P disruption budgets. It actually controls um and and making sure that not all of your uh replicas are being consolidated at the same time."

Strategic Instance Selection for Node Pools [3:43]

Choosing the right instances for a node pool involves giving Carpenter the right amount of flexibility and a broader choice of instance types.
Diversification: Having more instance types and sizes across different families, generations, and sizes theoretically allows Carpenter to find a better cost-performance fit.
Avoiding Extremes:
- Too Small Instances: These cause network overhead, add complexity to scheduling, and increase monitoring challenges.
- Too Large Nodes: If not fully utilized, large nodes become a waste of money and make consolidation less efficient.
The decision of what is "too small" or "too big" is dependent on the specific cluster and use case.

"So in theory, the more instance uh instant types and and sizes you uh you give carpenter across different uh families, generations and sizes, so in theory it can find the better uh better fit."

Managing DaemonSets for Cost Efficiency [5:31]

DaemonSets are containers for system services (monitoring, logging, telemetry) that run on each node, contributing to overhead.
Larger Nodes: A small overhead of DaemonSets per node.
Many Small Nodes: A larger overhead of DaemonSets, consuming more resources.
This overhead can silently inflate costs, especially if using software licensed per node.

"So when we have larger nodes we have small overhead of demon sets and on the opposite if we have a lot of small nodes it means we have a large overhead of um of demon sets."

Leveraging Spot Instances for Cost Savings [6:17]

Carpenter simplifies the management of spot instances by allowing them in node pools.
It automatically identifies spot instances with the best price and availability, chooses suitable machines, and handles interruptions by moving pods to other locations.
Suitable Workloads: Any workload that can tolerate interruptions, such as web servers behind load balancers, stateless microservices, batch jobs, and dev/staging environments.
Best Practice: Configure spot instances with a failover to on-demand. Carpenter will attempt to provision a spot instance first and transparently move to on-demand if capacity is unavailable.
Separate Node Pools: Creating distinct node pools for spot and on-demand instances allows for the use of node affinity in deployments to control the distribution of pods.

"And the good news is that Carpenter makes management of spot instances really easy."

Utilizing Graviton Instances and Availability Zones [8:34]

Graviton Instances: Offer comparable or better performance than traditional instances at a lower price. Most modern applications can run on ARM architectures with no code changes. The primary investment is building Docker images for both ARM and x86 architectures.
Cross-Availability Zone Traffic: This traffic is expensive. For non-production environments, limiting node pools to a specific availability zone can enforce data locality and significantly reduce networking costs. This is not recommended for production.
Failover Configuration: It's possible to configure failover mechanisms using weights and affinity rules to utilize another zone if a primary zone experiences a failure.

"As you probably know, Graviton provides at least the same performance or even better for a lower price and most of the modern applications are able to run on ARM architectures with no code change."

Ongoing Challenges and Zesti's Solution [10:12]

Connecting Metrics to Costs: Kubernetes provides many metrics, but it's difficult to directly link them to costs and identify inefficiencies, especially in large, multi-cluster environments.
Infrastructure Lag: Even with Carpenter's instant decisions, the underlying infrastructure can take minutes to be ready for containers.
Operational Complexity: Constantly modifying and adjusting setups for true efficiency requires ongoing effort.
Zesti's Platform: Offers continuous automation, allowing teams to focus on building rather than optimizing.
- Deep Visibility: Provides insights into cluster and container usage and utilization to identify inefficiencies.
- Predictive Models: Make scaling decisions more efficient.
- Automated Actions: Executes optimizations on behalf of teams.
- Workload Resizing: Automatically resizes workloads vertically and horizontally, including persistent volumes.
- Resource Optimization Layer: Actively right-sizes and distributes workloads.
- Financial Optimization Layer: Navigates and distributes workloads between spot instances and savings plans to reduce on-demand costs.
- Fast Scale Technology: Maintains hibernated nodes pre-cached and ready in 30 seconds for instant recovery from spot instance failures, enabling more workloads to use spot instances with less risk.
Zesti claims to provide at least 70% cost savings by combining resource and cloud financial optimization practices.