The Cloud Native Computing Foundation (CNCF) released Kubernetes 1.28, named Planternetes. The release has new features such as mixed version proxy, support for CDI injection into containers, and sidecar container awareness. In addition, beta features are included in the release, such as ValidatingAdmissionPolicies, match conditions for admission webhooks, and enabling swap space in Linux.
In version 1.28, CephFS in-tree plugin is deprecated, and CSI migration for GCE PD is removed. Several features have been marked generally available or stable, such as recovery from non-graceful node shutdown, automatic and retroactive assignment of default StorageClass, and changes to support skew between control plane and node versions.
The new release has a new package repository for Debian and RPM at pkgs.k8.io, replacing the legacy package repositories at apt.kubernetes.io and yum.kubernetes.io. Such repositories are deprecated and frozen from September 13, 2023. Users can check if they are affected by reading the blog post on the project’s website to take the necessary steps.
Three high-severity vulnerabilities (CVE-2023-3676, CVE-2023-3893, and CVE-2023-3955) have been addressed and patched in this release. They are related to insufficient input sanitization that can lead to privilege escalation on Windows nodes.
A mixed version proxy support has been introduced to enable the kube-apiserver to respond to requests even when control plane components are at different versions. This is done by having the kube-apiserver check which kube-apiserver can serve the requested resource and then proxy it to a suitable peer, which can be helpful during maintenance.
Native sidecar containers have also been introduced in this release. A restartPolicy field for init containers is presented with the only valid value Always to mark an init container as a sidecar. Unlike prior versions, in 1.28, the Kubelet will only wait for the init container to have started without completion before starting the main containers of the Pod.
ValidatingAdmissionPolicies has graduated to beta, allowing declarative and in-process alternatives to validating admission webhooks. Such policies use the Common Expression Language (CEL) to describe the validation rules of a policy. This is turned off by default.
In addition, support for enabling swap memory space on Linux graduated to beta. This feature enables the Kubelet to support Kubernetes workloads on Linux to use worker nodes swap space. This can benefit administrators looking for node-level performance tuning or developers with applications using swap memory.
Recovery from non-graceful node shutdown became generally available in this release. This allows stateful workloads to restart on a different node in case a node shuts down suddenly or is in an irrecoverable state due to an operating system crash or hardware failure.
Changes to support skew between control plane and node components versions were flagged as stable in version 1.28. Such modifications allow the control plane components, including the kube-apiserver to be three minor (n-3) versions apart from node components, including the Kubelet and kube-proxy. This enables end users to perform minor version upgrades to nodes once a year while staying within upstream support.
In summary, Kubernetes version 1.28 has 45 enhancements, including 19 entering alpha, 12 becoming generally available or stable, and 14 graduating to beta. In addition, 3 features are being deprecated or removed according to the release notes, CNCF held a webinar on September 6, 2023, to discuss the changes from the release team.