This topic explains how to enable Apigee hybrid.

Overview

Workload Identity is a way for applications running within GKE (Google Kubernetes Engine) to access Google Cloud services. For a good overview of Workload Identity, see:

• Introducing Workload Identity: Better authentication for your GKE applications
• Using Workload Identity

A Google Cloud IAM service account is an identity that an application can use to make requests to Google APIs. These service accounts are referred to as GSA (Google Service Accounts) in the document. For more information aobut GSAs, see Service accounts.

Separately, Kubernetes also has the concept of service accounts. A service account provides an identity for processes that run in a Pod. Kubernetes service accounts are Kubernetes resources, while Google service accounts are specific to Google Cloud. For information on Kubernetes service accounts, see Configure Service Accounts for Pods in the Kubernetes documentation.

With Apigee hybrid 1.4 and later, Apigee creates and uses a Kubernetes service account for each type of component. Enabling Workload Identity allows the hybrid components to interract with the Kubernetes service accounts.

Prerequisites

Before enabling Workload Identity for Apigee hybrid, Workload Identity must be enabled for the GKE cluster running Apigee.

If you have followed the instructions for Anthos Service Mesh (ASM), Workload Identity is already enabled for the cluster.

You can confirm if Workload Identity is enabled by running the following command:

gcloud container clusters describe $CLUSTER_NAME

The output should include something like the following:

…
…
status: RUNNING
subnetwork: default
workloadIdentityConfig:
  workloadPool: PROJECT_ID.svc.id.goog

When running Apigee hybrid on GKE, the standard practice is to create and download private keys (.json files) for each of the service accounts. When using Workload Identity, you do not need to download service account private keys and add them to GKE clusters.

Enable Workload Identity

These instructions are divided into three sections:

• Prepare to enable Workload Identity. Follow these instructions to update Node pools and initialize variables prior to enabling Workload Identity.
• Enable Workload Identity for a fresh install. Follow these instructions for a new Apigee hybrid installation or if you have not already installed ASM on your hybrid installation.
• Upgrade an installation to use Workload Identity. Follow these instructions to enable Workload Identity on your existing Apigee hybrid installation.

Prepare to enable Workload Identity

The following instructions describe how to enable Workload Identity for Apigee hybrid runtime.

Update node pools

Ensure Workload Identity is enabled per node pool with the following command:

gcloud container node-pools update $NODE_POOL_NAME \
  --cluster=$CLUSTER_NAME \
  --workload-metadata=GKE_METADATA

Initialize variables

Initialize the following variables:

export PROJECT_ID=my-project-id
export ORG_NAME=$PROJECT_ID
export ENV_NAME=my-environment-name
export NAMESPACE=apigee #the namespace where apigee is installed

gcloud config set project $PROJECT_ID

Enabling Workload Identity for a fresh install

1. Add the highlighted line below to your overrides.yaml file under the gcp stanza:

   gcp:
     projectID: "my-project"
     name: "my-project"
     region: "us-west1"
     workloadIdentityEnabled: true

2. Create Google Service Accounts. There are two methods you can use:
   • Use the create-service-account tool to create a Google service account per component with the following command:

     $APIGEECTL_HOME/tools/create-service-account --env prod --dir $APIGEECTL_HOME/../service-accounts

     This command will create the following service accounts:

     • apigee-logger
     • apigee-metrics
     • apigee-cassandra
     • apigee-udca
     • apigee-synchronizer
     • apigee-mart
     • apigee-watcher
     • apigee-distributed-trace

     The following instructions assume you used the create-service-account tool to generate the service accounts.

   • Define custom naming conventions and service accounts per environment (for advanced users)
3. Create the Kubernetes Service Accounts.

   For org level components:

   • Cassandra

     kubectl create sa -n $NAMESPACE apigee-cassandra-schema-setup-$(apigeectl encode --org $ORG_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-cassandra,org=$ORG_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-cassandra-schema-setup-$(apigeectl encode --org $ORG_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com

     kubectl create sa -n $NAMESPACE apigee-cassandra-user-setup-$(apigeectl encode --org $ORG_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-cassandra,org=$ORG_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-cassandra-user-setup-$(apigeectl encode --org $ORG_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com

   • MART

     kubectl create sa -n $NAMESPACE apigee-mart-$(apigeectl encode --org $ORG_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-mart,org=$ORG_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-mart-$(apigeectl encode --org $ORG_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com

   • Apigee Connect

     kubectl create sa -n $NAMESPACE apigee-connect-$(apigeectl encode --org $ORG_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-connect,org=$ORG_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-connect-$(apigeectl encode --org $ORG_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com
     

   • Apigee Watcher

     kubectl create sa -n $NAMESPACE apigee-watcher-$(apigeectl encode --org $ORG_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-watcher,org=$ORG_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-watcher-$(apigeectl encode --org $ORG_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com

   For each environment:

   • Runtime

     kubectl create sa -n $NAMESPACE apigee-runtime-$(apigeectl encode --org $ORG_NAME --env $ENV_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-runtime,org=$ORG_NAME,env=$ENV_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-runtime-$(apigeectl encode --org $ORG_NAME --env $ENV_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com

   • UDCA

     kubectl create sa -n $NAMESPACE apigee-udca-$(apigeectl encode --org $ORG_NAME --env $ENV_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-udca,org=$ORG_NAME,emv=$ENV_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-udca-$(apigeectl encode --org $ORG_NAME --env $ENV_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com

   • Synchronizer

     kubectl create sa -n $NAMESPACE apigee-synchronizer-$(apigeectl encode --org $ORG_NAME --env $ENV_NAME)-sa \
       && kubectl label sa -n $NAMESPACE -l app=apigee-synchronizer,org=$ORG_NAME,env=$ENV_NAME \
       && kubectl annotate serviceaccount \
       --namespace $NAMESPACE apigee-syncrhonizer-$(apigeectl encode --org $ORG_NAME --env $ENV_NAME)-sa iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com

4. Continue installing Apigee hybrid as you normally would.

Upgrading an install to use Workload Identity

Here is a sample showing Google Service Accounts (GSA) created for Apigee:

gcloud iam service-accounts list | grep apigee

apigee-connect   apigee-connect@$PROJECT_ID.iam.gserviceaccount.com    False
apigee-runtime   apigee-runtime@$PROJECT_ID.iam.gserviceaccount.com    False
apigee-metrics   apigee-metrics@$PROJECT_ID.iam.gserviceaccount.com    False
apigee-mart      apigee-mart@$PROJECT_ID.iam.gserviceaccount.com       False
apigee-watcher   apigee-watcher@$PROJECT_ID.iam.gserviceaccount.com    False
apigee-sync      apigee-sync@$PROJECT_ID.iam.gserviceaccount.com       False
apigee-udca      apigee-udca@$PROJECT_ID.iam.gserviceaccount.com       False

Here is a sample Kubernetes Service Accounts (KSA) created for Apigee (assumes Apigee hybrid 1.4 or higher is installed):

kubectl get sa -n $NAMESPACE

apigee-cassandra-schema-setup-ORG_NAME-cb84b88-sa                1         xxd
apigee-cassandra-user-setup-ORG_NAME-cb84b88-sa                  1         xxd
apigee-connect-agent-ORG_NAME-cb84b88-sa                         1         xxd
apigee-init                                                        1         xxd
apigee-mart-ORG_NAME-cb84b88-sa                                  1         xxd
apigee-metrics-apigee-telemetry                                    1         xxd
apigee-runtime-ORG_NAME-ENV_NAME-1d0dc5e-sa                    1         xxd
apigee-synchronizer-ORG_NAME-ENV_NAME-1d0dc5e-sa               1         xxd
apigee-udca-ORG_NAME-ENV_NAME-1d0dc5e-sa                       1         xxd
apigee-watcher-ORG_NAME-cb84b88                                  1         xxd

1. Add the Workload Identity role to each service account:

   gcloud iam service-accounts add-iam-policy-binding \
     --role roles/iam.workloadIdentityUser \
     --member "serviceAccount:$PROJECT_ID.svc.id.goog[$NAMESPACE/$KSA_NAME]" \
     GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com
   

   For example, if you were setting the permissions for Apigee Synchronizer, you would run:

   export KSA_NAME=$(kubectl get sa -n apigee -l app=apigee-synchronizer,env=$ENV_NAME,org=$ORG_NAME --output=jsonpath={.items..metadata.name})
   

   gcloud iam service-accounts add-iam-policy-binding --role roles/iam.workloadIdentityUser --member "serviceAccount:$PROJECT_ID.svc.id.goog[apigee/$KSA_NAME]" apigee-sync@$PROJECT_ID.iam.gserviceaccount.com
   

2. Annotate each KSA with the GSA details:

   kubectl annotate serviceaccount \
     --namespace $NAMESPACE \
     $KSA_NAME \
    iam.gke.io/gcp-service-account=GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com
   

   For example, if you were setting the permissions for Apigee Synchronizer, you would run:

   export KSA_NAME=$(kubectl get sa -n apigee -l app=apigee-synchronizer,env=$ENV_NAME,org=$ORG_NAME --output=jsonpath={.items..metadata.name})
   

   kubectl annotate serviceaccount --namespace $NAMESPACE $KSA_NAME iam.gke.io/gcp-service-account=apigee-sync@$PROJECT_ID.iam.gserviceaccount.com
   

3. Validate if the steps worked:

   gcloud config set project $PROJECT_ID
   

   kubectl run --rm -it --image google/cloud-sdk:slim --serviceaccount $KSA_NAME --namespace $NAMESPACE workload-identity-test -- gcloud auth list
   

   If you don't see a command prompt, try pressing Enter.

   If the steps were correctly run, you should see a response like the following:

                      Credentialed Accounts
   ACTIVE  ACCOUNT
   *       GSA@PROJECT_ID.iam.gserviceaccount.com

4. If upgrading from a previous install, clean up secrets that contained service account private keys:

   kubectl delete secrets -n $NAMESPACE $(k get secrets -n $NAMESPACE | grep svc-account | awk '{print $1}')
   

5. Check logs:

   kubectl logs -n $NAMESPACE -l app=apigee=synchronizer,env=$ENV_NAME,org=$ORG_NAME apigee-synchronizer