Migrate nodes to Linux cgroupv2

Autopilot Standard

Starting with version 1.32, Google Kubernetes Engine (GKE) migrates clusters running cgroupv1 to cgroupv2. This page instructs you how to do the following:

Check which cgroup mode your cluster's nodes are running, and if your workloads might be affected by transitioning between cgroup modes.
Migrate GKE Standard cluster node pools to cgroupv2.
Temporarily opt out of GKE automatically migrating node pools using cgroupv1 to cgroupv2. Follow these instructions if your cluster runs workloads which might be affected by transitioning between cgroup modes.

You can manually migrate or temporarily opt out Standard node pools. You can't migrate or temporarily opt out Autopilot clusters.

You can skip reading this page if you know that your workloads run as expected on cgroupv2, or aren't impacted by cgroup mode configuration. GKE automatically migrates clusters running cgroupv1 to cgroupv2 with version 1.32 and later.

About Linux control groups

The kubelet and the container runtime use Linux kernel control groups (cgroups) for resource management, such as limiting how much CPU or memory each container in a Pod can access. There are two modes of the cgroup subsystem in the kernel: cgroupv1 and cgroupv2. Kubernetes support for cgroupv2 was introduced as alpha in Kubernetes version 1.18, beta in 1.22, and GA in 1.25. For more details, refer to the Kubernetes cgroups v2 documentation.

To learn about configuring a cgroup mode for Standard clusters, see Linux cgroup mode configuration options.

How GKE is transitioning to `cgroupv2`

Review the following timeline to understand how GKE is transitioning existing clusters to use cgroupv2:

For versions earlier than 1.26, cgroupv1 was the default for nodes. For versions 1.26 or later, cgroupv2 is the default for new nodes. There is no change to existing nodes. To learn more about which cgroup mode your GKE clusters run by default, see Check the cgroup mode of cluster nodes.
With minor version 1.31, GKE deprecates cgroupv1.
Starting with version 1.32, GKE migrates clusters running cgroupv1 to cgroupv2. You can temporarily prevent this automatic migration by explicitly specifying that a node pool use cgroupv1.
With minor version 1.34, GKE removes support for cgroupv1.

For approximate timing of automatic upgrades to later minor versions such as 1.31 and 1.32, see the Estimated schedule for release channels.

Before you begin

Before you start, make sure you have performed the following tasks:

Enable the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.
Note: For existing gcloud CLI installations, make sure to set the compute/region and compute/zone properties. By setting default locations, you can avoid errors in gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location.

Check the cgroup mode of cluster nodes

The default cgroup mode depends on the type of cluster or node pool, and the version. With version 1.26 or later, the default is cgroupv2. With version 1.25 or earlier, the default is cgroupv1:

For Autopilot clusters and new Standard cluster node pools created with node auto-provisioning, the cgroup mode is based on the initial cluster version.
For Standard cluster node pools manually created without node auto-provisioning, the cgroup mode is based on the initial node pool version.

To check the cgroup mode, follow the instructions based on your cluster mode.

Autopilot

Run the following command, replacing CLUSTER_NAME with the name of your cluster:

gcloud container clusters describe CLUSTER_NAME \
    --format='value(initialClusterVersion)'

GKE Autopilot clusters that were initially created with GKE version 1.25 or earlier run cgroupv1. Clusters initially created with version 1.26 or later run cgroupv2.

Standard

With GKE Standard clusters, the cgroup mode is set at the node pool level. To check the mode for individual node pools, follow the instructions to Verify cgroup configuration. If your cluster's nodes are already using cgroupv2, no further action is necessary.

Migrate nodes to `cgroupv2`

We recommend that you migrate existing nodes before GKE automatically migrates them in version 1.32 or later. You can manually migrate Standard node pools. You can't migrate Autopilot clusters.

Do the following steps to migrate nodes running cgroupv1:

Check the cgroup mode of the nodes. If your cluster's nodes are already using cgroupv2, no further action is necessary.
Review the considerations about the migration, Migrating to cgroup v2, to ensure that your workloads are prepared to use the new version of the API.
Migrate the nodes for your Standard cluster by adding the following to the node system configuration for each node pool:
```
  linuxConfig:
  cgroupMode: 'CGROUP_MODE_V2'
```

Temporarily opt out of automatic migration to `cgroupv2`

You can temporarily opt out Standard node pools. You can't temporarily opt out Autopilot clusters.

To temporarily avoid automatic migration of Standard cluster node pools running cgroupv1 to cgroupv2 with minor versions 1.32 and later, you must explicitly set cgroupv1.

For existing Standard node pools, add the following to the node system configuration for each node pool:

linuxConfig:
  cgroupMode: 'CGROUP_MODE_V1'

To learn more, see Customizing node system configuration.