Expose an ingress gateway using an external load balancer

Overview

With Cloud Service Mesh you can deploy and manage ingress gateways as part of your service mesh. You can further configure load balancing for your cluster with Cloud Service Mesh by using external load balancers (physical or software load balancers outside of the cluster) to send traffic to the ingress gateway.

This page shows you how to configure an external load balancer with Cloud Service Mesh. Alternatively, you can set up ingress with multiple backend configs.

Before you begin

To complete the steps in this document you need the following resources:

  • A Kubernetes cluster with Cloud Service Mesh installed.

  • An external load balancer that can access the nodes where your cluster is running. You will configure this external load balancer to front the ingress gateway of your cluster via the External IP Address.

Set up your environment

Run the following commands from a workstation that can access the cluster you intend to use. Make sure that the kubectl tool is configured to use the cluster context specific to your cluster.

  1. Set the environment variables.

    export ASM_INGRESSGATEWAY_NAMESPACE=asm-ingressgateway
export ASM_INGRESSGATEWAY_DEPLOYMENT_NAME=asm-ingressgateway
export ASM_INGRESSGATEWAY_SERVICE_NAME=asm-ingressgateway
export ASM_INGRESSGATEWAY_NODE_LABEL=asm-ingressgateway

  2. Set the IP address of the external load balancer.

    export EXTERNAL_LB_IP_ADDRESS=EXTERNAL_LB_IP_ADDRESS

  3. [Optional] Label the ingress gateway nodes. This ensures that the gateway gets deployed to specific nodes in the cluster.

    kubectl label nodes INGRESSGATEWAY_NODE_IP ${ASM_INGRESSGATEWAY_NODE_LABEL}=
    • INGRESSGATEWAY_NODE_IP: is the node(s) in your Kubernetes cluster that hosts the ingress gateway. Run this kubectl command for as many ingress nodes you have.

Create the ingress gateway

  1. Create the namespace. This namespace will be used to deploy the ingress gateway.

    kubectl create namespace ${ASM_INGRESSGATEWAY_NAMESPACE}

  2. Enable the namespace for injection. The steps depend on your Cloud Service Mesh type (either managed or in-cluster).

    Managed

    1. Use the following command to locate the available release channels:

      kubectl -n istio-system get controlplanerevision

      The output is similar to the following:

      NAME                AGE
asm-managed         6d7h
asm-managed-rapid   6d7h

      In the output, the value under the NAME column is the revision label that corresponds to the available release channel for the Cloud Service Mesh version.

    2. Apply the revision label to the namespace:

      kubectl label namespace ${ASM_INGRESSGATEWAY_NAMESPACE} \
  istio-injection- istio.io/rev=REVISION_LABEL --overwrite

    In-cluster

    1. Use the following command to locate the revision label on istiod:

      kubectl get deploy -n istio-system -l app=istiod -o \
  jsonpath={.items[*].metadata.labels.'istio\.io\/rev'}'{"\n"}'

    2. Apply the revision label to the namespace. In the following command, REVISION is the value of the istiod revision label that you noted in the previous step.

      kubectl label namespace ${ASM_INGRESSGATEWAY_NAMESPACE} \
  istio-injection- istio.io/rev=REVISION --overwrite

  3. Apply the ingress gateway manifest file.

    kubectl --namespace ${ASM_INGRESSGATEWAY_NAMESPACE} apply --filename https://raw.githubusercontent.com/GoogleCloudPlatform/anthos-service-mesh-samples/main/docs/ingress-gateway-external-lb/ingress-gateway.yaml

    Expected output:

    serviceaccount/asm-ingressgateway created
role.rbac.authorization.k8s.io/asm-ingressgateway created
rolebinding.rbac.authorization.k8s.io/asm-ingressgateway created
deployment.apps/asm-ingressgateway created
service/asm-ingressgateway created
poddisruptionbudget.policy/asm-ingressgateway created
horizontalpodautoscaler.autoscaling/asm-ingressgateway created

  4. Patch the ingressgateway service with the external load balancer IP address.

    cat <<EOF > asm-external-ip-patch.yaml
spec:
  externalIPs:
    - ${EXTERNAL_LB_IP_ADDRESS}
  loadBalancerIP: ${EXTERNAL_LB_IP_ADDRESS}
EOF

kubectl --namespace ${ASM_INGRESSGATEWAY_NAMESPACE} patch service/${ASM_INGRESSGATEWAY_SERVICE_NAME} --patch "$(cat asm-external-ip-patch.yaml)"

  5. [Optional] Patch the ingressgateway deployment for the ingress gateway nodes label affinity.

    cat <<EOF > asm-ingress-node-label-patch.yaml
spec:
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: ${ASM_INGRESSGATEWAY_NODE_LABEL}
                operator: Exists
EOF

kubectl --namespace ${ASM_INGRESSGATEWAY_NAMESPACE} patch deployment/${ASM_INGRESSGATEWAY_DEPLOYMENT_NAME} --patch "$(cat asm-ingress-node-label-patch.yaml)"

Set up the external load balancer

In this section, you will configure the external load balancer to connect with the ingress gateway from the cluster.

Fetch ingress gateway Service port information

  1. Get the NodePorts.

    export HTTP_INGRESS_PORT=$(kubectl --namespace ${ASM_INGRESSGATEWAY_NAMESPACE} get service/${ASM_INGRESSGATEWAY_SERVICE_NAME} --output jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
export HTTPS_INGRESS_PORT=$(kubectl --namespace ${ASM_INGRESSGATEWAY_NAMESPACE} get service/${ASM_INGRESSGATEWAY_SERVICE_NAME} --output jsonpath='{.spec.ports[?(@.name=="https")].nodePort}')
export STATUS_PORT=$(kubectl --namespace ${ASM_INGRESSGATEWAY_NAMESPACE} get service/${ASM_INGRESSGATEWAY_SERVICE_NAME} --output jsonpath='{.spec.ports[?(@.name=="status-port")].nodePort}')

  2. Display the NodePorts.

    echo HTTP_INGRESS_PORT=${HTTP_INGRESS_PORT}
echo HTTPS_INGRESS_PORT=${HTTPS_INGRESS_PORT}
echo STATUS_PORT=${STATUS_PORT}

Configure the external load balancer

Use the NodePorts displayed in the previous step to configure connectivity between the external load balancer and the ingress gateway.

  1. Configure the health check in your load balancer configuration.

    hosts:    CLUSTER_NODE_IP
Protocol: HTTP
Port:     STATUS_PORT
Path:     /healthz/ready

    • CLUSTER_NODE_IP: is the IP address of the nodes in your Kubernetes cluster that hosts the ingress gateway. This IP address must be reachable from your external load balancer. You may have to set up this configuration multiple times, once per cluster node.

    • STATUS_PORT: is the NodePort via which the ingress gateway's health status API is exposed. You can copy this information from the previous step. It will be the same for every node in the cluster.

  2. Configure node pools in your load balancer for routing HTTP and HTTPS traffic. Use the following IP:PORT configuration for traffic on port 80 (HTTP) and port 443 (HTTPS).

    80  ->  CLUSTER_NODE_IP:HTTP_INGRESS_PORT
443 ->  CLUSTER_NODE_IP:HTTPS_INGRESS_PORT

    • CLUSTER_NODE_IP: is the IP address of the nodes in your Kubernetes cluster that hosts the ingress gateway. This IP address must be reachable from your external load balancer. You may have to set up this configuration multiple times, once per cluster node.

    • HTTP_INGRESS_PORT: is the NodePort via which the ingress gateway's HTTP traffic is exposed. You can copy this information from the previous step. It will be the same for every node in the cluster.

    • HTTPS_INGRESS_PORT: is the NodePort via which the ingress gateway's HTTPS traffic is exposed. You can copy this information from the previous step. It will be the same for every node in the cluster.

To verify your set up, ensure that the health checks on your load balancer are passing.

What's next