Cluster trust

This page describes the trust in Google Kubernetes Engine (GKE) clusters, including how control planes and nodes authenticate requests.

Intracluster communication

There are many connections in a cluster for communication between Kubernetes components.

Control plane to node

The control plane communicates with a node for managing containers. When the control plane sends a request, for example, kubectl logs, to nodes, the request is sent over a Konnectivity proxy tunnel. Requests sent by the control plane are protected with TLS, providing authentication, integrity, and encryption. When a node sends a request to the control plane, for example, from the kubelet to API server, that request is authenticated and encrypted using mutual TLS.

All newly created and updated clusters use TLS 1.3 for control plane to node communication. TLS 1.2 is the minimum supported version for control plane to node communication.

Node to node

Nodes are Compute Engine VMs for which connections inside of the Google Cloud production network are authenticated and encrypted. For details, refer to the VM-to-VM section of the Encryption in Transit whitepaper.

Pod to Pod

When a Pod sends a request to another Pod, that traffic isn't authenticated by default. GKE encrypts traffic between Pods on different nodes at the network layer. Traffic between Pods on the same node isn't encrypted by default. For details about the network-layer encryption, see Google Cloud virtual network encryption and authentication.

You can restrict Pod-to-Pod traffic with a NetworkPolicy, and you can encrypt all Pod-to-Pod traffic, including traffic on the same node, by using a service mesh like Cloud Service Mesh or a different method of application-layer encryption.

etcd to etcd

An instance of etcd may communicate with another instance of etcd to keep state updated. When an instance of etcd sends a request to another instance, that request is authenticated and encrypted using mutual TLS. The traffic never leaves a GKE-owned network protected by firewalls.

Control plane to etcd

This communication is encrypted using mutual TLS.

Root of trust

GKE has the following configuration:

  • The cluster root Certificate Authority (CA) is used to validate the API server and kubelets' client certificates. That is, control planes and nodes have the same root of trust. Any kubelet within the cluster node pool can request a certificate from this CA using the certificates.k8s.io API, by submitting a certificate signing request. The root CA has a limited lifetime as described in the Cluster root CA lifetime section.
  • A separate per-cluster etcd CA is used to validate etcd's certificates.

API server and kubelets

The API server and the kubelets rely on the cluster root CA for trust. In GKE, the control plane API certificate is signed by the cluster root CA. Each cluster runs its own CA, so that if one cluster's CA is compromised, no other cluster CA is affected.

An internal service manages root keys for this CA, which are non-exportable. This service accepts certificate signing requests, including those from the kubelets in each GKE cluster. Even if the API server in a cluster were compromised, the CA would not be compromised, so no other clusters would be affected.

Each node in the cluster is injected with a shared secret at creation, which it can use to submit certificate signing requests to the cluster root CA and obtain kubelet client certificates. These certificates are then used by the kubelet to authenticate its requests to the API server. This shared secret is reachable by Pods on the node, unless you enable Shielded GKE Nodes or Workload Identity Federation for GKE.

Cluster root CA lifetime

The cluster root CA has a limited lifetime, after which any certificates signed by the expired CA are invalid. Check the approximate expiry date of your cluster's CA by following the instructions in Check credential lifetime.

You should manually rotate your credentials before your existing root CA expires. If the CA expires and you don't rotate your credentials, your cluster might enter an unrecoverable state. GKE has the following mechanisms to try and prevent unrecoverable clusters:

  • Your cluster enters a DEGRADED state seven days before CA expiry

  • GKE attempts an automatic credential rotation 30 days before CA expiry. This automatic rotation ignores maintenance windows and might cause disruptions as GKE recreates nodes to use new credentials. External clients, like kubectl in local environments, won't work until you update them to use the new credentials.

To learn how to perform a rotation, see Rotate your cluster credentials.

etcd

In GKE, etcd relies on a separate per-cluster etcd CA for trust.

Root keys for the etcd CA are distributed to the metadata of each virtual machine (VM) on which the control plane runs. Any code executing on control plane VMs, or with access to compute metadata for these VMs, can sign certificates as this CA. Even if etcd in a cluster were compromised, the CA is not shared between clusters, so no other clusters would be affected.

The etcd certificates are valid for five years.

Rotating your certificates

To rotate all your cluster's API server and kubelet certificates, perform a credential rotation. There is no way for you to trigger a rotation of the etcd certificates; this is managed for you in GKE.

What's next