Configure kernel parameters for AlloyDB Omni on Kubernetes

This page provides information about setting up Kubernetes nodes that host AlloyDB Omni database clusters for the optimal performance of the AlloyDB Omni Kubernetes operator and the AlloyDB Omni database engine.

Configuring kernel parameters lets AlloyDB Omni use system memory and IO resources more efficiently when handling heavy workloads.

Prerequisite

Before you start, make sure your Kubernetes nodes run Linux kernel 6.1 or higher, specifically a kernel that supports the MADV_COLLAPSE and MADV_POPULATE_WRITE flags. For more information about these flags, see Linux madwise documentation.

Apply recommended kernel settings to Kubernetes nodes

The following table lists required kernel parameters and their corresponding values for Kubernetes nodes that run database cluster pods:

To configure the kernel parameters on your Kubernetes nodes using a DaemonSet, do the following:

1. Apply labels to the nodes where you plan to run database clusters using instructions in Add a label to a node.

2. To set the kernel parameters on the nodes labeled LABEL_KEY=LABEL_VALUE, create a DaemonSet:

   cat << EOF | kubectl apply -f -
   apiVersion: apps/v1
   kind: DaemonSet
   metadata:
     name: alloydb-omni-kernel-tuning
     namespace: DS_NAMESPACE
   spec:
     selector:
       matchLabels:
         name: alloydb-omni-kernel-tuning
     template:
       metadata:
         labels:
           name: alloydb-omni-kernel-tuning
       spec:
         volumes:
           - name: host-sys
             hostPath:
               path: /sys
         nodeSelector:
           LABEL_KEY: "LABEL_VALUE"
         restartPolicy: Always
         terminationGracePeriodSeconds: 1
         initContainers:
           - name: enable-thp-mmc
             image: INIT_IMAGE
             volumeMounts:
               - name: host-sys
                 mountPath: /sys
             securityContext:
               privileged: true
             command: ["sh", "-c", "sysctl -w vm.max_map_count=MAX_MAP_COUNT && echo within_size >/sys/kernel/mm/transparent_hugepage/shmem_enabled"]
         containers:
           - name: CONTAINER_NAME
             image: CONTAINER_IMAGE
             command: ["watch", "-n", "600", "cat", "/sys/kernel/mm/transparent_hugepage/shmem_enabled"]
   EOF
   

   Replace the following:

   • DS_NAMESPACE: the namespace where you want to deploy the DaemonSet—for example, default.
   • LABEL_KEY: the identifier for the label—for example, workload.
   • LABEL_VALUE: the data associated with the identifier for the label—for example, database.
   • INIT_IMAGE: the name of the image used for the init container.
   • MAX_MAP_COUNT: the maximum number of memory map areas a process can create—for example, 1073741824.
   • CONTAINER_NAME: the name of the main container—for example, main.
   • CONTAINER_IMAGE: the name of the image used for the main container—for example, latest.

3. After deploying the DaemonSet, to verify that all pods in your Kubernetes cluster are in the Running state and that you have one pod for each node with the label LABEL_KEY=LABEL_VALUE, use the following command:

   kubectl get pods -l LABEL_KEY=LABEL_VALUE
   

   An example output looks as follows:

   NAME                               READY   STATUS    RESTARTS   AGE
   alloydb-omni-kernel-tuning-2dkwh   1/1     Running   0          22s
   alloydb-omni-kernel-tuning-pgkbj   1/1     Running   0          19s
   

4. To verify the kernel flag is successfully applied, for each of the pods identified after deploying the DaemonSet, run the following command:

   kubectl logs POD_NAME
   

   Replace POD_NAME with the name of the pod you want to examine the logs of.

   An example output looks as follows:

   always [within_size] advise never deny force
   

Run database clusters on Kubernetes nodes with recommended kernel parameters

To deploy database clusters on Kubernetes nodes with recommended kernel parameters, add a nodeAffinity section to one of the following sections of your database cluster manifest:

• primarySpec.schedulingConfig for primary instances
• spec.schedulingConfig for read pool instances

      nodeaffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: LABEL_KEY
              operator: In
              values:
              - "LABEL_VALUE"

For more information about running database clusters on specific Kubernetes nodes, see Assign nodes to a database cluster using scheduling.

Verify the application of kernel parameters

To verify kernel parameters are applied to database pods, do the following:

1. If high availability is enabled, specifically spec.availability.numberOfStandbys is set to a value greater than zero, verify that the database custom resource shows DBClusterReady in the DBCLUSTERPHASE column and True in the HAREADYSTATUS column.

   kubectl get dbcluster -n DBCLUSTER_NAMESPACE DBCLUSTER_NAME
   

   Replace the following:

   • DBCLUSTER_NAME: the name of the database cluster you verify.
   • DBCLUSTER_NAMESPACE: the name of the specific namespace where your database cluster resides.

   An example output looks as follows:

   NAME               PRIMARYENDPOINT   PRIMARYPHASE   DBCLUSTERPHASE   HAREADYSTATUS   HAREADYREASON
   dbcluster-sample   10.29.21.240      Ready          DBClusterReady   True            Ready
   

2. List Kubernetes pods that belong to the database cluster:

   kubectl get pods -n DBCLUSTER_NAMESPACE -l alloydbomni.internal.dbadmin.goog/dbcluster=DBCLUSTER_NAME
   

3. To check shared memory information, run the following command:

   kubectl exec -n DBCLUSTER_NAMESPACE POD_NAME -- grep Shmem /proc/meminfo
   

4. The output must contain numbers other than zero in all of the entries.

   An example output looks as follows:

   Shmem:          126255872 kB
   ShmemHugePages: 18403328 kB
   ShmemPmdMapped:  2961408 kB
   

   If you see 0 kB shows in any of the entries, restart the corresponding pod:

   kubectl delete pod -n DBCLUSTER_NAMESPACE POD_NAME
   

5. Repeat steps 1 through 5 after the pod is in the Running state.