Amazon Web Services (AWS) has released version 1.0 of Karpenter, an open-source Kubernetes cluster auto-scaling tool. This release marks Karpenter's graduation from beta status and introduces stable APIs and several new features.
Karpenter, initially launched in November 2021, has evolved into a comprehensive Kubernetes-native node lifecycle manager. The project has gained popularity among industry leaders such as Slack as we reported earlier this year, and has become one of the top ten most popular AWS open-source projects on GitHub.
In a post for Infracloud, Himanshu Verma explains how Karpenter eliminates the need for pre-configured Node Groups and resource sizing, and directly manages instances using Launch Templates. Karpenter automatically provisions new nodes as needed, responding to unschedulable pods in real-time. By dynamically adjusting cluster compute capacity, it optimizes both performance and cost-effectiveness.
The 1.0 release brings many fixes and several new features. The key features of Karpenter 1.0 include:
- Enhanced disruption controls: Users can now set disruption budgets by reason, allowing finer-grained control over node disruptions for scenarios such as underutilization, emptiness, or drift.
- Renamed consolidation policy: The "WhenUnderutilized" policy has been renamed to "WhenEmptyOrUnderutilized" for clarity.
- New consolidation control: A "consolidateAfter" parameter allows users to specify how long Karpenter waits before consolidating nodes when pods are added or removed.
- Termination grace period: A new "terminationGracePeriod" setting gives cluster administrators more control over node lifetime and graceful termination.
- Drift feature promotion: The drift feature, which replaces nodes that deviate from a desired state, is now enabled by default.
- AMI selection changes: The "amiSelectorTerms" field is now required, and a new "alias" term lets users select specific versions of Amazon EKS-optimized AMIs.
- Instance Metadata Service (IMDS) access restrictions: To improve security, pod access to the IMDS is now blocked by default.
- Kubelet configuration moved: Kubelet settings have been relocated to the EC2NodeClass API.
- NodeClaims made immutable: NodeClaims can no longer be modified after creation to align with expected behaviour.
- NodePool nodeClassRef changes: All nodeClassRef fields are now required, and the "apiVersion" field has been renamed to "group".
- Metric changes: Several Prometheus metrics have been modified, which might require users to update their dashboards.
There are also some planned deprecations, including removing certain annotations and ConfigMap-based configuration.
For users looking to upgrade from beta versions, AWS provides a migration path using Kubernetes webhook conversion, provided they already use a version with the latest APIs. This process allows for in-place API upgrades without requiring node replacement. However, users are advised to review the complete migration documentation and test the upgrade process in a non-production environment before proceeding.
In a video walkthrough on X (formerly Twitter), Olalekan Taofeek explained in detail how he upgraded Karpenter from 0.37 to 1.0 - a version jump bigger than that provided for by the migration path. He highlighted that there were several breaking changes that required careful attention:
- The user data is no longer used for Kubernetes configuration.
- Changes to the node pool/group configuration - the API version and field names have changed, requiring updates to the user's configurations.
- Addition of a new "startup time" field that needs to be accounted for in the user's workloads.
- Relocation of some fields from the node pool to the node class configuration.
The news of Karpenter 1.0 has been well received, with Vatsal Bajpai, for example writing on X (formerly Twitter):
"Karpenter has become so essential for k8s on cloud, happy to see this!"
Writing in a thread on Reddit, users generally appreciated the release, with some pointing out that a port is also available for Azure, though some also lamented the lack of GCP support. Others pondered whether the Karpenter functionality could be fully integrated into the EKS control plane.
For more information or to provide feedback, users can reach out through the Kubernetes Slack #karpenter channel or on GitHub.