Run your own certificate authorities and keys in GKE

Autopilot Standard

This page shows you how to configure your Google Kubernetes Engine (GKE) cluster control plane with certificate authorities (CAs) and keys that you manage. This guidance is intended for security administrators who have specific organizational compliance or policy requirements for control over credential issuance and signing.

This page describes one part of a set of optional control plane features in GKE that lets you perform tasks like verifying your control plane security posture or configuring encryption and credential signing in the control plane using keys that you manage. For details, see About GKE control plane authority.

By default, Google Cloud applies various security measures to the managed control plane. This page describes optional capabilities that give you more visibility or control over the GKE control plane.

You should already be familiar with the following concepts:

Control plane credential components

GKE clusters use specific CAs and keys to issue credentials in the cluster, like X.509 certificates or ServiceAccount tokens. You can create keys in Cloud Key Management Service (Cloud KMS) and CAs in Certificate Authority Service (CA Service) and configure your clusters to use these resources instead of Google Cloud-managed CAs and keys.

To learn more about the specific components that you create, see Self-managed CAs and keys.

Usage with other GKE control plane authority features

GKE control plane authority provides the following features related to self-managed keys:

Run your own certificate authorities (CAs) and keys (this page)
Encrypt control plane components

Objectives

Create keys in Cloud KMS
Create CAs in CA Service
Grant Identity and Access Management (IAM) roles to the GKE service agent
Create a GKE cluster that uses your CAs and keys
Verify that the cluster uses your CAs and keys

Costs

In this document, you use the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish the tasks that are described in this document, you can avoid continued billing by deleting the resources that you created. For more information, see Clean up.

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Install the Google Cloud CLI.

To initialize the gcloud CLI, run the following command:

gcloud init

Note: If you installed the gcloud CLI previously, make sure you have the latest version by running

gcloud components
      update

Make sure that billing is enabled for your Google Cloud project.

Enable the Kubernetes Engine, Certificate Authority Service, and Cloud Key Management Service APIs:

gcloud services enable container.googleapis.com privateca.googleapis.com cloudkms.googleapis.com

Install the Google Cloud CLI.

To initialize the gcloud CLI, run the following command:

gcloud init

Note: If you installed the gcloud CLI previously, make sure you have the latest version by running

gcloud components
      update

Make sure that billing is enabled for your Google Cloud project.

Enable the Kubernetes Engine, Certificate Authority Service, and Cloud Key Management Service APIs:

gcloud services enable container.googleapis.com privateca.googleapis.com cloudkms.googleapis.com

Ensure that your environment is eligible to use GKE control plane authority features. To opt-in to these features, contact your Google Cloud sales team.
To reliably track credential issuance and usage, ensure that the following Data Access audit logs are enabled:
- Cloud KMS: DATA_READ
- CA Service: ADMIN_READ and ADMIN_WRITE
To enable these log types, see Enable Data Access audit logs.

Required roles and permissions

To get the permissions that you need to run your own CAs and keys, ask your administrator to grant you the following IAM roles:

Create Cloud KMS keys: Cloud KMS Admin (roles/cloudkms.admin) on your key project
Create CA pools and root CAs: CA Service Admin (roles/privateca.admin) on your key project
Create GKE clusters: Kubernetes Engine Cluster Admin (roles/container.clusterAdmin) on your cluster project

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Requirements

You must use GKE version 1.31.1-gke.1846000 or later.

Limitations

The following limitations apply:

You can only use keys from Cloud KMS. You can't use another KMS provider or another encryption provider.
Cloud External Key Manager (Cloud EKM) keys aren't supported.
You can only use CAs from CA Service.

Prepare the environment

In this section, you identify Google Cloud projects that you'll use in this tutorial, and you create a key ring in Cloud KMS to hold your keys.

Identify projects

We recommend that you use separate Google Cloud projects as follows:

Key project: contains all keys and CAs.
Cluster project: contains your GKE clusters.

You can optionally use the same project for your keys, CAs, and GKE clusters, but we recommend that you use separate projects so that the teams that manage cryptographic operations in your organization are separated from the teams that manage cluster operations.

Create a key ring

Create a key ring in the key project to hold all of the keys for a specific cluster. You must create the key ring in the same location as your GKE cluster.

Run the following command:

gcloud kms keyrings create KEY_RING_NAME \
    --location=us-central1 \
    --project=KEY_PROJECT_ID

Replace the following:

KEY_RING_NAME: a name for your key ring.
KEY_PROJECT_ID: the project ID of your key project.

Create keys

For each of the credential authorities like service account keys and CAs, you create a key using Cloud KMS. This section shows you how to create the keys that GKE uses to sign and verify credentials in the cluster. You can specify your own properties for these keys depending on your organization's needs. For details, see the Create a key page and the projects.locations.keyRings.cryptoKeys API reference.

Consider the following when you create these resources in Cloud KMS:

If you have a existing key ring in your key project, you can use that key ring to store all of the keys that you create for use with your cluster.
Your key ring must be in the same Google Cloud location as your cluster to minimize latency.
Keys must specify asymmetric-signing as the key purpose.
Use the following algorithms based on the type of key:
- ServiceAccount signing keys: a strong RSA signing PKCS1 algorithm, like rsa-sign-pkcs1-4096-sha256 or rsa-sign-pkcs1-3072-sha256.
- Certificate authority keys: a strong algorithm like ec-sign-p256-sha256.
Cloud HSM hardware keys are supported, but the software protection level is sufficient for most use cases. For details about hardware keys, see Cloud HSM.
Don't modify the default duration for key destruction.
GKE doesn't prevent you from deleting Cloud KMS keys, including CA Service keys, that are in use by the cluster. Before you delete keys or CAs, ensure that the resources are unused.

To create your keys, run the following commands:

Create the Kubernetes ServiceAccount signing key, which you also specify as the service account verification key during cluster creation:

gcloud kms keys create sa-signing-key \
    --keyring=KEY_RING_NAME \
    --location=us-central1\
    --purpose="asymmetric-signing" \
    --protection-level=hsm \
    --default-algorithm=rsa-sign-pkcs1-4096-sha256 \
    --project=KEY_PROJECT_ID

Replace KEY_PROJECT_ID with the project ID for your dedicated key project.

Create the cluster root CA key:

gcloud kms keys create cluster-ca-key \
    --keyring=KEY_RING_NAME \
    --location=us-central1\
    --purpose="asymmetric-signing" \
    --protection-level=hsm \
    --default-algorithm=ec-sign-p256-sha256 \
    --project=KEY_PROJECT_ID

Create the etcd peer root CA key:

gcloud kms keys create etcd-peer-ca-key \
    --keyring=KEY_RING_NAME \
    --location=us-central1\
    --purpose="asymmetric-signing" \
    --protection-level=hsm \
    --default-algorithm=ec-sign-p256-sha256 \
    --project=KEY_PROJECT_ID

Create the etcd API root CA key:

gcloud kms keys create etcd-api-ca-key \
    --keyring=KEY_RING_NAME \
    --location=us-central1\
    --purpose="asymmetric-signing" \
    --protection-level=hsm \
    --default-algorithm=ec-sign-p256-sha256 \
    --project=KEY_PROJECT_ID

Create the aggregation root CA key:

gcloud kms keys create aggregation-ca-key \
    --keyring=KEY_RING_NAME \
    --location=us-central1\
    --purpose="asymmetric-signing" \
    --protection-level=hsm \
    --default-algorithm=ec-sign-p256-sha256 \
    --project=KEY_PROJECT_ID

Create the CAs

After creating the keys for each of the control plane functions, use each key to create the CA pools and corresponding root CAs using CA Service:

Create the cluster CA pool:
```
gcloud privateca pools create cluster-ca-pool \
    --location=us-central1 \
    --tier=enterprise \
    --project=KEY_PROJECT_ID \
    --no-publish-crl --no-publish-ca-cert
```
The --no-publish-crl flag and the --no-publish-ca-cert flag are optional. Omitting these flags publishes certificates to a Cloud Storage bucket. For details, see Enable CA certificate and CRL publication for CAs in a CA pool.

Create the cluster root CA:

gcloud privateca roots create cluster-root-ca \
    --pool=cluster-ca-pool \
    --location=us-central1 \
    --kms-key-version=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/cluster-ca-key/cryptoKeyVersions/1 \
    --subject="CN=cluster-ca, O=ORGANIZATION" \
    --project=KEY_PROJECT_ID \
    --auto-enable

Replace ORGANIZATION with your organization name.

Create the etcd peer CA pool:

gcloud privateca pools create etcd-peer-ca-pool \
    --location=us-central1 \
    --tier=enterprise \
    --project=KEY_PROJECT_ID \
    --no-publish-crl --no-publish-ca-cert

Create the etcd peer root CA:

gcloud privateca roots create etcd-peer-root-ca \
    --pool=etcd-peer-ca-pool \
    --location=us-central1 \
    --kms-key-version=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/etcd-peer-ca-key/cryptoKeyVersions/1 \
    --subject="CN=etcd-peer-ca, O=ORGANIZATION" \
    --project=KEY_PROJECT_ID \
    --auto-enable

Create the etcd API CA pool:

gcloud privateca pools create etcd-api-ca-pool \
    --location=us-central1 \
    --tier=enterprise \
    --project=KEY_PROJECT_ID \
    --no-publish-crl --no-publish-ca-cert

Create the etcd API root CA:

gcloud privateca roots create etcd-api-root-ca \
    --pool=etcd-api-ca-pool \
    --location=us-central1 \
    --kms-key-version=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/etcd-api-ca-key/cryptoKeyVersions/1 \
    --subject="CN=etcd-api-ca, O=ORGANIZATION" \
    --project=KEY_PROJECT_ID \
    --auto-enable

Create the aggregation CA pool:

gcloud privateca pools create aggregation-ca-pool \
    --location=us-central1 \
    --tier=enterprise \
    --project=KEY_PROJECT_ID \
    --no-publish-crl --no-publish-ca-cert

Create the aggregation root CA:

gcloud privateca roots create aggregation-root-ca \
    --pool=aggregation-ca-pool \
    --location=us-central1 \
    --kms-key-version=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/aggregation-ca-key/cryptoKeyVersions/1 \
    --subject="CN=aggregation-ca, O=ORGANIZATION" \
    --project=KEY_PROJECT_ID \
    --auto-enable

Grant IAM roles to the GKE service agent

The GKE service agent requires access to the resources that you created in Cloud KMS and in CA Service. The service agent uses these resources to sign, verify, and issue credentials in the cluster. You can use the following predefined IAM roles:

Kubernetes Engine KMS Crypto Key User (roles/container.cloudKmsKeyUser)
CA Service Certificate Manager (roles/privateca.certificateManager)

To grant these roles to the GKE service agent, do the following:

Find the project number of your cluster project:
```
gcloud projects describe CLUSTER_PROJECT_ID \
    --format='value(projectNumber)'
```
Replace CLUSTER_PROJECT_ID with the project ID of your cluster project.

Grant the Kubernetes Engine KMS Crypto Key User role on the service account signing key that you created in Create keys:

gcloud kms keys add-iam-policy-binding sa-signing-key \
  --location=us-central1 \
  --keyring=KEY_RING_NAME \
  --member="serviceAccount:service-CLUSTER_PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com" \
  --role=roles/container.cloudKmsKeyUser \
  --project=KEY_PROJECT_ID

Replace CLUSTER_PROJECT_NUMBER with the project number of the cluster project.

Grant the CA Service Certificate Manager role on the CA pools that you created in Create the CAs:

gcloud privateca pools add-iam-policy-binding cluster-ca-pool \
    --location=us-central1 \
    --member="serviceAccount:service-CLUSTER_PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com" \
    --role=roles/privateca.certificateManager \
    --project=KEY_PROJECT_ID

gcloud privateca pools add-iam-policy-binding etcd-peer-ca-pool \
    --location=us-central1 \
    --member="serviceAccount:service-CLUSTER_PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com" \
    --role=roles/privateca.certificateManager \
    --project=KEY_PROJECT_ID

gcloud privateca pools add-iam-policy-binding etcd-api-ca-pool \
    --location=us-central1 \
    --member="serviceAccount:service-CLUSTER_PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com" \
    --role=roles/privateca.certificateManager \
    --project=KEY_PROJECT_ID

gcloud privateca pools add-iam-policy-binding aggregation-ca-pool \
    --location=us-central1 \
    --member="serviceAccount:service-CLUSTER_PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com" \
    --role=roles/privateca.certificateManager \
    --project=KEY_PROJECT_ID

Set up CAs and keys on a new cluster

After you create keys, CA pools, root CAs, and grant IAM roles to the GKE service agent, create a new cluster that uses these resources.

The flags that you specify in the cluster creation command require the following resource paths as values:

Path to a key version in Cloud KMS for the service account signing key that you created in Create keys. You specify this path for the service-account-signing-keys flag and for the service-account-verification-keys flag.
Path to each of the CA pools that you created in Create the CAs.

To configure a new cluster to use your keys and CAs, do the following steps:

Find the path to the latest enabled service account signing key version:

gcloud kms keys versions list \
    --key=sa-signing-key \
    --keyring=KEY_RING_NAME \
    --location=us-central1 \
    --project=KEY_PROJECT_ID \
    --filter="STATE=ENABLED" --sort-by=~ --format="value(name)" | sed 1q

Replace KEY_PROJECT_ID with the project ID of the key project.

The output is similar to the following:

projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1

Find the paths to each of the CA pools that you created:

gcloud privateca pools list --format="get(name)" \
    --project=KEY_PROJECT_ID

The output is similar to the following:

projects/KEY_PROJECT_ID/locations/us-central1/caPools/cluster-ca-pool
projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-peer-ca-pool
projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-api-ca-pool
projects/KEY_PROJECT_ID/locations/us-central1/caPools/aggregation-ca-pool

Ensure that the output contains all of the CA pools that you created for GKE.

Create a cluster

In this section, you create a cluster with different options specified depending on which GKE control plane authority features you want to configure. You can only configure these features on a cluster during cluster creation. The following commands create Autopilot mode clusters. To create Standard mode clusters instead, use the same flags with the gcloud container clusters create command.

To only configure the CAs and keys that you created in this tutorial, run the following command:

gcloud container clusters create-auto example-cluster \
    --location=us-central1 \
    --project=CLUSTER_PROJECT_ID \
    --cluster-version=VERSION \
    --service-account-signing-keys=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1 \
    --service-account-verification-keys=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1 \
    --cluster-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/cluster-ca-pool \
    --etcd-peer-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-peer-ca-pool \
    --etcd-api-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-api-ca-pool \
    --aggregation-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/aggregation-ca-pool

Replace the following:

CLUSTER_PROJECT_ID: the project ID of the cluster project.
VERSION: the GKE version of the cluster. Must be 1.31.1-gke.1846000 or later.

To configure the CAs and keys as well as control plane boot disk encryption and etcd encryption, do the following:

Perform all of the key configuration steps in Encrypt etcd and control plane boot disks.
Find the paths to each of the keys by using the instructions in Use encryption keys in a cluster.

Create a cluster:

gcloud container clusters create-auto example-cluster \
    --location=us-central1 \
    --project=CLUSTER_PROJECT_ID \
    --cluster-version=VERSION \
    --service-account-signing-keys=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1 \
    --service-account-verification-keys=projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1 \
    --cluster-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/cluster-ca-pool \
    --etcd-peer-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-peer-ca-pool \
    --etcd-api-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-api-ca-pool \
    --aggregation-ca=projects/KEY_PROJECT_ID/locations/us-central1/caPools/aggregation-ca-pool \
    --control-plane-disk-encryption-key=PATH_TO_DISK_KEY \
    --gkeops-etcd-backup-encryption-key=PATH_TO_ETCD_BACKUP_KEY

Replace the following:

CLUSTER_PROJECT_ID: the project ID of the cluster project.
VERSION: the GKE version of the cluster. Must be 1.31.1-gke.1846000 or later.
PATH_TO_DISK_KEY: the path to your disk encryption key.
PATH_TO_ETCD_BACKUP_KEY: the path to your etcd internal backup encryption key.

You can also use these flags when you create a new Standard mode cluster.

Verify that the cluster uses your specified keys and CAs

This section shows you how to verify the keys and CAs that were used during cluster creation. You can perform this verification using Cloud Logging or by using the Google Cloud CLI.

Use Logging to verify keys and CAs

To verify the keys and CAs using Logging, do the following:

In the Google Cloud console, go to the Logs Explorer page:

Go to Logs Explorer

Specify the following query:

resource.type="gke_cluster"
resource.labels.cluster_name="CLUSTER_NAME"
resource.labels.location="CLUSTER_LOCATION"
protoPayload.serviceName="container.googleapis.com"
protoPayload.methodName=~"google.container.v(1|1alpha1|1beta1).ClusterManager.CreateCluster"
protoPayload.request.cluster.userManagedKeysConfig:*

protoPayload.request.cluster.userManagedKeysConfig:* filters the results for cluster creation logs that include keys and CAs that you manage.

Click Run query.

In the results, expand your cluster creation log. Verify that the paths to keys and CAs are the same as those that you created for that cluster, like in the following example:

# lines omitted for clarity
userManagedKeysConfig: {
  aggregationCa: "projects/KEY_PROJECT_ID/locations/us-central1/caPools/aggregation-ca-pool"
  clusterCa: "projects/KEY_PROJECT_ID/locations/us-central1/caPools/cluster-ca-pool"
  etcdApiCa: "projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-api-ca-pool"
  etcdPeerCa: "projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-peer-ca-pool"
  serviceAccountSigningKeys: [
    0: "projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1"
  ]
  serviceAccountVerificationKeys: [
    0: "projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1"
  ]
}

Use the gcloud CLI to verify keys and CAs

To verify that the cluster uses the CAs and keys that you created, run the following command:

gcloud container clusters describe example-cluster \
    --location=us-central1 \
    --project=CLUSTER_PROJECT_ID

The output should include the userManagedKeysConfig field like in the following example:

# lines omitted for clarity
userManagedKeysConfig:
  sa-signing-key: projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1
  sa-verification-key: projects/KEY_PROJECT_ID/locations/us-central1/keyRings/KEY_RING_NAME/cryptoKeys/sa-signing-key/cryptoKeyVersions/1
  cluster-ca: projects/KEY_PROJECT_ID/locations/us-central1/caPools/cluster-ca-pool
  etcd-peer-ca: projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-peer-ca-pool
  etcd-api-ca: projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-api-ca-pool
  aggregation-ca: projects/KEY_PROJECT_ID/locations/us-central1/caPools/aggregation-ca-pool

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Delete the projects

Delete a Google Cloud project:

gcloud projects delete PROJECT_ID

Delete individual resources

Delete the cluster:

gcloud container clusters delete example-cluster \
    --location=us-central1 \
    --project=CLUSTER_PROJECT_ID

Disable the root CAs:

gcloud privateca roots disable cluster-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/cluster-ca-pool \
    --project=KEY_PROJECT_ID

gcloud privateca roots disable etcd-peer-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-peer-ca-pool \
    --project=KEY_PROJECT_ID

gcloud privateca roots disable etcd-api-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-api-ca-pool \
    --project=KEY_PROJECT_ID

gcloud privateca roots disable aggregation-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/aggregation-ca-pool \
    --project=KEY_PROJECT_ID

Delete the root CAs:

gcloud privateca roots delete cluster-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/cluster-ca-pool \
    --project=KEY_PROJECT_ID

gcloud privateca roots delete etcd-peer-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-peer-ca-pool \
    --project=KEY_PROJECT_ID

gcloud privateca roots delete etcd-api-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/etcd-api-ca-pool \
    --project=KEY_PROJECT_ID

gcloud privateca roots delete aggregation-root-ca \
    --location=us-central1 \
    --pool=projects/KEY_PROJECT_ID/locations/us-central1/caPools/aggregation-ca-pool \
    --project=KEY_PROJECT_ID

Delete the CA pools:

gcloud privateca pools delete cluster-ca-pool --location=us-central1 \
    --project=KEY_PROJECT_ID

gcloud privateca pools delete etcd-peer-ca-pool --location=us-central1 \
    --project=KEY_PROJECT_ID

gcloud privateca pools delete etcd-api-ca-pool --location=us-central1 \
    --project=KEY_PROJECT_ID

gcloud privateca pools delete aggregation-ca-pool --location=us-central1 \
    --project=KEY_PROJECT_ID

Delete the keys:

gcloud kms keys versions destroy 1 \
    --location=us-central1 \
    --keyring=KEY_RING_NAME \
    --key=sa-signing-key \
    --project=KEY_PROJECT_ID

gcloud kms keys versions destroy 1 \
    --location=us-central1 \
    --keyring=KEY_RING_NAME \
    --key=cluster-ca-key \
    --project=KEY_PROJECT_ID

gcloud kms keys versions destroy 1 \
    --location=us-central1 \
    --keyring=KEY_RING_NAME \
    --key=etcd-peer-ca-key \
    --project=KEY_PROJECT_ID

gcloud kms keys versions destroy 1 \
    --location=us-central1 \
    --keyring=KEY_RING_NAME \
    --key=etcd-api-ca-key \
    --project=KEY_PROJECT_ID

gcloud kms keys versions destroy 1 \
    --location=us-central1 \
    --keyring=KEY_RING_NAME \
    --key=aggregation-ca-key \
    --project=KEY_PROJECT_ID

You can't delete key rings from Cloud KMS. However, key rings don't incur additional costs.

What's next

Track identity usage from the moment of issuance
Explore reference architectures, diagrams, and best practices about Google Cloud. Take a look at our Cloud Architecture Center.