在 Google Cloud 上安装 Kf

本文档介绍如何设置 GKE 集群，以及如何在之后安装 Kf 及其依赖项。

准备工作

GKE 集群要求

Kf 要求

请参阅 Kf 依赖项和架构页面，查看并了解 Kf 中的组件的访问权限。

依赖项矩阵列出了具体版本。

启用对 Compute Engine 的支持

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  5. Make sure that billing is enabled for your Google Cloud project.

  6. 启用 Compute Engine API。

    启用 API

启用对 Artifact Registry 的支持

  1. 启用 Artifact Registry API。

    启用 Artifact Registry API

启用并配置 GKE

在开始之前，请确保您已执行以下任务：

  • 启用 Google Kubernetes Engine API。
    • 启用 Google Kubernetes Engine API
  • 如果您要使用 Google Cloud CLI 执行此任务，请安装并初始化 gcloud CLI。 如果您之前安装了 gcloud CLI，请运行 gcloud components update 以获取最新版本。

设置环境变量

Linux 和 Mac

export PROJECT_ID=YOUR_PROJECT_ID
export CLUSTER_PROJECT_ID=YOUR_PROJECT_ID
export CLUSTER_NAME=kf-cluster
export COMPUTE_ZONE=us-central1-a
export COMPUTE_REGION=us-central1
export CLUSTER_LOCATION=${COMPUTE_ZONE} # Replace ZONE with REGION to switch
export NODE_COUNT=4
export MACHINE_TYPE=e2-standard-4
export NETWORK=default

Windows PowerShell

Set-Variable -Name PROJECT_ID -Value YOUR_PROJECT_ID
Set-Variable -Name CLUSTER_PROJECT_ID -Value YOUR_PROJECT_ID
Set-Variable -Name CLUSTER_NAME -Value kf-cluster
Set-Variable -Name COMPUTE_ZONE -Value us-central1-a
Set-Variable -Name COMPUTE_REGION -Value us-central1
Set-Variable -Name CLUSTER_LOCATION -Value $COMPUTE_ZONE # Replace ZONE with REGION to switch
Set-Variable -Name NODE_COUNT -Value 4
Set-Variable -Name MACHINE_TYPE -Value e2-standard-4
Set-Variable -Name NETWORK -Value default

设置服务账号

创建一个通过 Workload Identity 与 Kubernetes 服务账号关联的 Google Cloud 服务账号。这样可以避免创建和注入服务账号密钥。

  1. 创建 Kf 将使用的服务账号。

    gcloud iam service-accounts create ${CLUSTER_NAME}-sa \
--project=${CLUSTER_PROJECT_ID} \
--description="GSA for Kf ${CLUSTER_NAME}" \
--display-name="${CLUSTER_NAME}"

  2. 创建新的自定义 IAM 角色。

    gcloud iam roles create serviceAccountUpdater \
--project=${CLUSTER_PROJECT_ID} \
--title "Service Account Updater" \
--description "This role only updates members on a GSA" \
--permissions iam.serviceAccounts.get,iam.serviceAccounts.getIamPolicy,iam.serviceAccounts.list,iam.serviceAccounts.setIamPolicy

  3. 允许服务账号修改自己的政策。Kf 控制器将使用它来向政策添加新（名称）空间，从而重复使用 Workload Identity。

    gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
  --member="serviceAccount:${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
  --role="projects/${CLUSTER_PROJECT_ID}/roles/serviceAccountUpdater"

  4. 为监控指标授予角色，以便对 Cloud Monitoring 进行写入访问。

    gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
  --member="serviceAccount:${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/monitoring.metricWriter"

  5. 授予日志记录角色以提供 Cloud Logging 的写入权限。

    gcloud projects add-iam-policy-binding ${CLUSTER_PROJECT_ID} \
  --member="serviceAccount:${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/logging.logWriter"

创建 GKE 集群

gcloud container clusters create ${CLUSTER_NAME} \
  --project=${CLUSTER_PROJECT_ID} \
  --zone=${CLUSTER_LOCATION} \
  --num-nodes=${NODE_COUNT} \
  --machine-type=${MACHINE_TYPE} \
  --disk-size "122" \
  --network=${NETWORK} \
  --addons HorizontalPodAutoscaling,HttpLoadBalancing,GcePersistentDiskCsiDriver \
  --enable-dataplane-v2 \
  --enable-stackdriver-kubernetes \
  --enable-ip-alias \
  --enable-autorepair \
  --enable-autoupgrade \
  --scopes cloud-platform \
  --release-channel=regular \
  --workload-pool="${CLUSTER_PROJECT_ID}.svc.id.goog" \
  --service-account="${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com"

设置防火墙规则

Kf 需要打开一些防火墙端口。主节点需要能够在端口 80、443、8080、8443 和 6443 上与 pod 通信。

启用 Workload Identity

现在您已经拥有服务账号和 GKE 集群，接下来将集群的身份命名空间与集群关联。

gcloud iam service-accounts add-iam-policy-binding \
  --project=${CLUSTER_PROJECT_ID} \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:${CLUSTER_PROJECT_ID}.svc.id.goog[kf/controller]" \
  "${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com"

gcloud iam service-accounts add-iam-policy-binding \
  --project=${CLUSTER_PROJECT_ID} \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:${CLUSTER_PROJECT_ID}.svc.id.goog[cnrm-system/cnrm-controller-manager]" \
  "${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com"

目标 GKE 集群

运行以下命令来配置 kubectl 命令行访问权限：

gcloud container clusters get-credentials ${CLUSTER_NAME} \
    --project=${CLUSTER_PROJECT_ID} \
    --zone=${CLUSTER_LOCATION}

创建 Artifact Registry 代码库

  1. 为要存储的容器映像创建 Artifact Registry。

    gcloud artifacts repositories create ${CLUSTER_NAME} \
  --project=${CLUSTER_PROJECT_ID} \
  --repository-format=docker \
  --location=${COMPUTE_REGION}

  2. 向 Artifact Registry 代码库授予服务账号权限。

    gcloud artifacts repositories add-iam-policy-binding ${CLUSTER_NAME} \
  --project=${CLUSTER_PROJECT_ID} \
  --location=${COMPUTE_REGION} \
  --member="serviceAccount:${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
  --role='roles/artifactregistry.writer'

在集群上安装软件依赖项

  1. 安装 Cloud Service Mesh v1.12。

    1. 按照 Cloud Service Mesh 安装指南（包括创建入站流量网关的步骤）操作。

  2. 安装 Config Connector。

    1. 下载所需的 Config Connector Operator tar 文件。

    2. 解压缩 tar 文件。

      tar zxvf release-bundle.tar.gz

    3. 在集群上安装 Config Connector Operator。

      kubectl apply -f operator-system/configconnector-operator.yaml

    4. 配置 Config Connector Operator。

      1. 将以下 YAML 复制到名为 configconnector.yaml 的文件中：

        # configconnector.yaml
apiVersion: core.cnrm.cloud.google.com/v1beta1
kind: ConfigConnector
metadata:
  # the name is restricted to ensure that there is only one
  # ConfigConnector resource installed in your cluster
  name: configconnector.core.cnrm.cloud.google.com
spec:
  mode: cluster
  googleServiceAccount: "KF_SERVICE_ACCOUNT_NAME" # Replace with the full service account resolved from ${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com

      2. 将配置应用到您的集群。

        kubectl apply -f configconnector.yaml

    5. 验证 Config Connector 是否已完全安装，然后再继续操作。

      • Config Connector 在命名空间 cnrm-system 中运行其所有组件。 通过运行以下命令验证 Pod 是否已准备就绪：

        kubectl wait -n cnrm-system --for=condition=Ready pod --all

      • 如果正确安装了 Config Connector，您应该会看到类似如下所示的输出：

        pod/cnrm-controller-manager-0 condition met
pod/cnrm-deletiondefender-0 condition met
pod/cnrm-resource-stats-recorder-86858dcdc5-6lqzb condition met
pod/cnrm-webhook-manager-58c799b8fb-kcznq condition met
pod/cnrm-webhook-manager-58c799b8fb-n2zpx condition met

    6. 设置 Workload Identity。

      kubectl annotate serviceaccount \
--namespace cnrm-system \
--overwrite \
cnrm-controller-manager \
iam.gke.io/gcp-service-account=${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com

  3. 安装 Tekton：

    kubectl apply -f "https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.29.0/release.yaml"

安装 Kf

  1. 安装 Kf CLI：

    Linux

    此命令会为系统上的所有用户安装 Kf CLI。请按照 Cloud Shell 标签页中的说明自行安装。

    gcloud storage cp gs://kf-releases/v2.7.3/kf-linux /tmp/kf
chmod a+x /tmp/kf
sudo mv /tmp/kf /usr/local/bin/kf

    Mac

    此命令会为系统上的所有用户安装 kf。

    gcloud storage cp gs://kf-releases/v2.7.3/kf-darwin /tmp/kf
chmod a+x /tmp/kf
sudo mv /tmp/kf /usr/local/bin/kf

    Cloud Shell

    如果您使用 bash，此命令会在 Cloud Shell 实例上安装 kf；您可能需要为其他 Shell 修改说明。

    mkdir -p ~/bin
gcloud storage cp gs://kf-releases/v2.7.3/kf-linux ~/bin/kf
chmod a+x ~/bin/kf
echo "export PATH=$HOME/bin:$PATH" >> ~/.bashrc
source ~/.bashrc

    Windows

    此命令会将 kf 下载到当前目录。如果要从当前目录以外的任何位置调用，请将其添加到路径中。

    gcloud storage cp gs://kf-releases/v2.7.3/kf-windows.exe kf.exe

  2. 安装 operator：

    kubectl apply -f "https://storage.googleapis.com/kf-releases/v2.7.3/operator.yaml"

  3. 为 Kf 配置 operator：

    kubectl apply -f "https://storage.googleapis.com/kf-releases/v2.7.3/kfsystem.yaml"

  4. 设置 Secret 和默认值：

    export CONTAINER_REGISTRY=${COMPUTE_REGION}-docker.pkg.dev/${CLUSTER_PROJECT_ID}/${CLUSTER_NAME}

kubectl patch \
kfsystem kfsystem \
--type='json' \
-p="[{'op': 'replace', 'path': '/spec/kf', 'value': {'enabled': true, 'config': {'spaceContainerRegistry': '${CONTAINER_REGISTRY}', 'secrets':{'workloadidentity':{'googleserviceaccount':'${CLUSTER_NAME}-sa', 'googleprojectid':'${CLUSTER_PROJECT_ID}'}}}}}]"

验证安装

  kf doctor --retries=20

清理

这些步骤应移除在创建并准备新的 GKE 集群部分中创建的所有组件。

  1. 删除 Google 服务账号：

    gcloud iam service-accounts delete ${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com

  2. 删除 IAM 政策绑定：

    gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
    --member="serviceAccount:${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
    --role="roles/iam.serviceAccountAdmin"
gcloud projects remove-iam-policy-binding ${CLUSTER_PROJECT_ID} \
    --member="serviceAccount:${CLUSTER_NAME}-sa@${CLUSTER_PROJECT_ID}.iam.gserviceaccount.com" \
    --role="roles/monitoring.metricWriter"

  3. 删除容器映像代码库：

    gcloud artifacts repositories delete ${CLUSTER_NAME} \
  --location=${COMPUTE_REGION}

  4. 卸载 Kf：

    kubectl patch kfsystem kfsystem \
  --type='json' \
  -p="[{'op': 'replace', 'path': '/spec/kf', 'value': {'enabled': false, }}]"

  5. （可选）删除 GKE 集群：

    gcloud container clusters delete ${CLUSTER_NAME} --zone ${CLUSTER_LOCATION}