設定 GKE Dataplane V2 觀測功能


本頁面說明如何設定 Google Kubernetes Engine (GKE) 叢集,以使用 GKE Dataplane V2 可觀測性功能,適用於 GKE 1.28 以上版本。如要進一步瞭解 GKE Dataplane V2 觀測功能的優點和需求,請參閱「關於 GKE Dataplane V2 觀測功能」。

事前準備

開始之前,請確認你已完成下列工作:

  • 啟用 Google Kubernetes Engine API。
  • 啟用 Google Kubernetes Engine API
  • 如要使用 Google Cloud CLI 執行這項工作,請安裝初始化 gcloud CLI。如果您先前已安裝 gcloud CLI,請執行 gcloud components update,取得最新版本。

設定 GKE Dataplane V2 指標

如要收集指標,請設定 GKE Dataplane V2 指標。建立叢集時,或更新以 GKE Dataplane V2 執行的叢集時,可以設定 GKE Dataplane V2 指標。您可以使用 gcloud CLI 啟用或停用 GKE Dataplane V2 指標。

建議您在 GKE 叢集上啟用 GKE Dataplane V2 指標和 Google Cloud Managed Service for Prometheus。啟用這兩項功能後,GKE Dataplane V2 指標就會傳送至 Google Cloud Managed Service for Prometheus。

建立啟用 GKE Dataplane V2 指標的 Autopilot 叢集

建立新的 GKE Autopilot 叢集時,GKE 會預設在叢集上啟用 GKE Dataplane V2 指標,不需要特定標記。

如要搭配使用 GKE Autopilot 叢集 GKE Dataplane V2 指標和 Google Cloud Managed Service for Prometheus,請設定ClusterPodMonitoring資源,以便擷取指標並傳送至 Google Cloud Managed Service for Prometheus。

  1. 建立 ClusterPodMonitoring 資訊清單:

    # Copyright 2023 Google LLC
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     https://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    apiVersion: monitoring.googleapis.com/v1
    kind: ClusterPodMonitoring
    metadata:
      name: advanced-datapath-observability-metrics
    spec:
      selector:
        matchLabels:
          k8s-app: cilium
      endpoints:
      - port: flowmetrics
        interval: 60s
        metricRelabeling:
        # only keep denormalized pod flow metrics
        - sourceLabels: [__name__]
          regex: 'pod_flow_(ingress|egress)_flows_count'
          action: keep
        # extract pod name
        - sourceLabels: [__name__, destination]
          regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${2}'
          targetLabel: pod_name
          action: replace
        - sourceLabels: [__name__, source]
          regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${2}'
          targetLabel: pod_name
          action: replace
        # extract workload name by removing 2 last "-XXX" parts
        - sourceLabels: [pod_name]
          regex: '([a-zA-Z0-9-\.]+)((-[a-zA-Z0-9\.]+){2})'
          replacement: '${1}'
          targetLabel: workload_name
          action: replace
        # extract workload name by removing one "-XXX" part when pod name has only 2 parts (eg. daemonset)
        - sourceLabels: [pod_name]
          regex: '([a-zA-Z0-9\.]+)((-[a-zA-Z0-9\.]+){1})'
          replacement: '${1}'
          targetLabel: workload_name
          action: replace
        # extract pod namespace
        - sourceLabels: [__name__, destination]
          regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${1}'
          targetLabel: namespace_name
          action: replace
        - sourceLabels: [__name__, source]
          regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${1}'
          targetLabel: namespace_name
          action: replace
        # extract remote workload name
        - sourceLabels: [__name__, source]
          regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${2}'
          targetLabel: remote_workload
          action: replace
        - sourceLabels: [__name__, destination]
          regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${2}'
          targetLabel: remote_workload
          action: replace
        # extract remote workload namespace
        - sourceLabels: [__name__, source]
          regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${1}'
          targetLabel: remote_namespace
          action: replace
        - sourceLabels: [__name__, destination]
          regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
          replacement: '${1}'
          targetLabel: remote_namespace
          action: replace
        # default remote workload class to "pod"
        - replacement: 'pod'
          targetLabel: remote_class
          action: replace
        # extract remote workload class from reserved identity
        - sourceLabels: [__name__, source]
          regex: 'pod_flow_ingress_flows_count;reserved:([^/]*)'
          replacement: '${1}'
          targetLabel: remote_class
          action: replace
        - sourceLabels: [__name__, destination]
          regex: 'pod_flow_egress_flows_count;reserved:([^/]*)'
          replacement: '${1}'
          targetLabel: remote_class
          action: replace
      targetLabels:
        metadata: []
    
  2. 套用 ClusterPodMonitoring 資訊清單:

    kubectl apply -f ClusterPodMonitoring.yaml
    

建立啟用 GKE Dataplane V2 指標的標準叢集

如要啟用 GKE Dataplane V2 指標,請使用 --enable-dataplane-v2-metrics 旗標建立叢集:

gcloud container clusters create CLUSTER_NAME \
    --enable-dataplane-v2 \
    --enable-ip-alias \
    --enable-managed-prometheus \
    --enable-dataplane-v2-metrics

更改下列內容:

  • CLUSTER_NAME:叢集名稱。

--enable-managed-prometheus 旗標會指示 GKE 使用 Google Cloud Managed Service for Prometheus 的指標。

在現有叢集上啟用 GKE Dataplane V2 指標

如要在現有叢集上啟用 GKE Dataplane V2 指標,請執行下列指令:

gcloud container clusters update CLUSTER_NAME \
    --enable-dataplane-v2-metrics

CLUSTER_NAME 替換為叢集名稱。

停用 GKE Dataplane V2 指標

如要停用 GKE Dataplane V2 指標,請按照下列步驟操作:

gcloud container clusters update CLUSTER_NAME \
    --disable-dataplane-v2-metrics

CLUSTER_NAME 替換為叢集名稱。

設定 GKE Dataplane V2 觀測工具

您可以使用私人端點存取 GKE Dataplane V2 觀測功能疑難排解工具。如要啟用 GKE Dataplane V2 觀測工具,您必須設定叢集,並啟用 GKE Dataplane V2。您可以在新叢集或現有叢集上啟用 GKE Dataplane V2 觀測工具。

建立已啟用可觀測性的 Autopilot 叢集

如要建立啟用 GKE Dataplane V2 可觀測性的 GKE Autopilot 叢集,請按照下列步驟操作:

gcloud container clusters create-auto CLUSTER_NAME \
    --enable-dataplane-v2-flow-observability \
    --location COMPUTE_LOCATION

請替換下列項目: * CLUSTER_NAME:叢集名稱。 * COMPUTE_LOCATION:叢集的 Compute Engine 位置

建立啟用觀測功能的標準叢集

如要建立啟用 GKE Dataplane V2 可觀測性的 GKE Standard 叢集,請按照下列步驟操作:

gcloud container clusters create CLUSTER_NAME \
    --enable-dataplane-v2 \
    --enable-ip-alias \
    --enable-dataplane-v2-flow-observability \
    --location COMPUTE_LOCATION

請替換下列項目: * CLUSTER_NAME:叢集名稱。 * COMPUTE_LOCATION:叢集的 Compute Engine 位置

在現有叢集上啟用 GKE Dataplane V2 觀測工具

如要在現有叢集上啟用 GKE Dataplane V2 可觀測性,請執行下列指令:

gcloud container clusters update CLUSTER_NAME \
    --enable-dataplane-v2-flow-observability \
    --location COMPUTE_LOCATION

更改下列內容:

停用 GKE Dataplane V2 觀測工具

如要停用現有叢集的 GKE Dataplane V2 可觀測性工具,請執行下列指令:

gcloud container clusters update CLUSTER_NAME \
    --disable-dataplane-v2-flow-observability

CLUSTER_NAME 替換為叢集名稱。

如何使用 Hubble CLI

啟用 GKE Dataplane V2 觀測功能後,即可在叢集上使用 Hubble CLI 工具。

  1. 定義 hubble-cli 二進位檔的別名:

    alias hubble="kubectl exec -it -n gke-managed-dpv2-observability deployment/hubble-relay -c hubble-cli -- hubble"
    
  2. 如要檢查 Hubble 狀態,請啟用 GKE Dataplane V2 觀測功能,並在所有 Autopilot 叢集中使用 Hubble CLI:

    hubble status
    
  3. 如要查看目前的流量,請使用 Hubble CLI,如下所示:

    hubble observe
    

如何部署 Hubble UI 二進位發行套件

啟用 GKE Dataplane V2 觀測功能後,即可部署開放原始碼 Hubble UI。

  1. 在 GKE 叢集中啟用可觀測性:

    1. 建立啟用可觀測性的 GKE 叢集:

      gcloud container clusters create-auto hubble-rc-auto \
          --location COMPUTE_LOCATION \
          --cluster-version VERSION \
          --enable-dataplane-v2-flow-observability
      

      更改下列內容:

    2. 或者,您也可以在現有叢集中啟用可觀測性:

      gcloud container clusters update CLUSTER_NAME \
          --location COMPUTE_LOCATION \
          --enable-dataplane-v2-flow-observability
      

      更改下列內容:

  2. 設定 kubectl 以連線至叢集:

    gcloud container clusters get-credentials CLUSTER_NAME \
        --location COMPUTE_LOCATION
    

    取代

  3. 部署 Hubble UI:

    # Copyright 2024 Google LLC
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     https://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: hubble-ui
      namespace: gke-managed-dpv2-observability
    ---
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: hubble-ui
      labels:
        app.kubernetes.io/part-of: cilium
    rules:
      - apiGroups:
          - networking.k8s.io
        resources:
          - networkpolicies
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - componentstatuses
          - endpoints
          - namespaces
          - nodes
          - pods
          - services
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - apiextensions.k8s.io
        resources:
          - customresourcedefinitions
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - cilium.io
        resources:
          - "*"
        verbs:
          - get
          - list
          - watch
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: hubble-ui
      labels:
        app.kubernetes.io/part-of: cilium
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: hubble-ui
    subjects:
      - kind: ServiceAccount
        name: hubble-ui
        namespace: gke-managed-dpv2-observability
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: hubble-ui-nginx
      namespace: gke-managed-dpv2-observability
    data:
      nginx.conf: |
        server {
            listen       8081;
            # uncomment for IPv6
            # listen       [::]:8081;
            server_name  localhost;
            root /app;
            index index.html;
            client_max_body_size 1G;
            location / {
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                # CORS
                add_header Access-Control-Allow-Methods "GET, POST, PUT, HEAD, DELETE, OPTIONS";
                add_header Access-Control-Allow-Origin *;
                add_header Access-Control-Max-Age 1728000;
                add_header Access-Control-Expose-Headers content-length,grpc-status,grpc-message;
                add_header Access-Control-Allow-Headers range,keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout;
                if ($request_method = OPTIONS) {
                    return 204;
                }
                # /CORS
                location /api {
                    proxy_http_version 1.1;
                    proxy_pass_request_headers on;
                    proxy_hide_header Access-Control-Allow-Origin;
                    proxy_pass http://127.0.0.1:8090;
                }
                location / {
                    # double `/index.html` is required here
                    try_files $uri $uri/ /index.html /index.html;
                }
            }
        }
    ---
    kind: Deployment
    apiVersion: apps/v1
    metadata:
      name: hubble-ui
      namespace: gke-managed-dpv2-observability
      labels:
        k8s-app: hubble-ui
        app.kubernetes.io/name: hubble-ui
        app.kubernetes.io/part-of: cilium
    spec:
      replicas: 1
      selector:
        matchLabels:
          k8s-app: hubble-ui
      template:
        metadata:
          labels:
            k8s-app: hubble-ui
            app.kubernetes.io/name: hubble-ui
            app.kubernetes.io/part-of: cilium
        spec:
          securityContext:
            fsGroup: 1000
            seccompProfile:
              type: RuntimeDefault
          serviceAccount: hubble-ui
          serviceAccountName: hubble-ui
          containers:
            - name: frontend
              image: quay.io/cilium/hubble-ui:v0.11.0
              ports:
                - name: http
                  containerPort: 8081
              volumeMounts:
                - name: hubble-ui-nginx-conf
                  mountPath: /etc/nginx/conf.d/default.conf
                  subPath: nginx.conf
                - name: tmp-dir
                  mountPath: /tmp
              terminationMessagePolicy: FallbackToLogsOnError
              securityContext:
                allowPrivilegeEscalation: false
                readOnlyRootFilesystem: true
                runAsUser: 1000
                runAsGroup: 1000
                capabilities:
                  drop:
                    - all
            - name: backend
              image: quay.io/cilium/hubble-ui-backend:v0.11.0
              env:
                - name: EVENTS_SERVER_PORT
                  value: "8090"
                - name: FLOWS_API_ADDR
                  value: "hubble-relay.gke-managed-dpv2-observability.svc:443"
                - name: TLS_TO_RELAY_ENABLED
                  value: "true"
                - name: TLS_RELAY_SERVER_NAME
                  value: relay.gke-managed-dpv2-observability.svc.cluster.local
                - name: TLS_RELAY_CA_CERT_FILES
                  value: /var/lib/hubble-ui/certs/hubble-relay-ca.crt
                - name: TLS_RELAY_CLIENT_CERT_FILE
                  value: /var/lib/hubble-ui/certs/client.crt
                - name: TLS_RELAY_CLIENT_KEY_FILE
                  value: /var/lib/hubble-ui/certs/client.key
              ports:
                - name: grpc
                  containerPort: 8090
              volumeMounts:
                - name: hubble-ui-client-certs
                  mountPath: /var/lib/hubble-ui/certs
                  readOnly: true
              terminationMessagePolicy: FallbackToLogsOnError
              securityContext:
                allowPrivilegeEscalation: false
                readOnlyRootFilesystem: true
                runAsUser: 1000
                runAsGroup: 1000
                capabilities:
                  drop:
                    - all
          volumes:
            - configMap:
                defaultMode: 420
                name: hubble-ui-nginx
              name: hubble-ui-nginx-conf
            - emptyDir: {}
              name: tmp-dir
            - name: hubble-ui-client-certs
              projected:
                # note: the leading zero means this number is in octal representation: do not remove it
                defaultMode: 0400
                sources:
                  - secret:
                      name: hubble-relay-client-certs
                      items:
                        - key: ca.crt
                          path: hubble-relay-ca.crt
                        - key: tls.crt
                          path: client.crt
                        - key: tls.key
                          path: client.key
    ---
    kind: Service
    apiVersion: v1
    metadata:
      name: hubble-ui
      namespace: gke-managed-dpv2-observability
      labels:
        k8s-app: hubble-ui
        app.kubernetes.io/name: hubble-ui
        app.kubernetes.io/part-of: cilium
    spec:
      type: ClusterIP
      selector:
        k8s-app: hubble-ui
      ports:
        - name: http
          port: 80
          targetPort: 8081
    
  4. 套用 hubble-ui-128.yaml 資訊清單:

    kubectl apply -f hubble-ui-128.yaml
    
  5. 透過通訊埠轉送公開 Service:

    kubectl -n gke-managed-dpv2-observability port-forward service/hubble-ui 16100:80 --address='0.0.0.0'
    
  6. 在網路瀏覽器中存取 Hubble UI:

    http://localhost:16100/

後續步驟