Encrypt metrics from your applications

All data, including metrics, must be encrypted in transit for security purposes. The Monitoring component of Google Distributed Cloud (GDC) air-gapped provides metrics in-transit encryption and authentication integration through the mutual Transport Layer Security (mTLS) method.

The mTLS configuration verifies that the parties at each end of a network connection have the correct certificate. This confirms that the parties are who they claim to be when communicating with each other.

This page explains how to set up mTLS for your metrics server by enabling mTLS metrics collection.

Before you begin

Before enabling metrics encryption, complete the following steps:

Ensure your metrics server is deployed.
Enable metrics collection.

Check that the MonitoringTarget custom resource shows a Ready status, for example:

apiVersion: monitoring.gdc.goog/v1
kind: MonitoringTarget
[...]
status:
  conditions:
  - lastTransitionTime: "2023-10-27T15:27:51Z"
    message: ""
    observedGeneration: 1
    reason: Ready
    status: True
    type: Ready

Query and view metrics on dashboards to confirm that the expected metrics from your metrics server are displayed.

Note: If your metrics don't show up, then your MonitoringTarget configuration has an issue that you must resolve before enabling mTLS authentication. Review the Collect metrics section to ensure that your metrics configuration is correct.

Mount the certificate

Your workload needs to use a certificate that you automatically generate when you enable mTLS on the MonitoringTarget custom resource.

You must add a volume and a volume mount for the automatically generated certificate. To do so, you can use the following template:

apiVersion: apps/v1
kind: Deployment # or StatefulSet
metadata:
  name: "SERVICE_NAME"
spec:
  template:
    spec:
      containers:
        - name: "CONTAINER_NAME"
          volumeMounts:
          - mountPath: "/etc/ssl/MONITORING_TARGET_NAME-cert"
            name: "MONITORING_TARGET_NAME-cert"
            readOnly: true
      volumes:
      - name: "MONITORING_TARGET_NAME-cert"
        secret:
          secretName: "MONITORING_TARGET_NAME-cert"

Replace the following:

SERVICE_NAME: the name of the Deployment or StatefulSet object.
CONTAINER_NAME: the name of the container where you are adding the volume mount.
MONITORING_TARGET_NAME: the name of the MonitoringTarget custom resource. The secret name is generated based on the MonitoringTarget name. Therefore, if the name of your MonitoringTarget custom resource is my-mon-target, then the secret name is my-mon-target-cert.

Serve metrics

Your metrics server now needs to serve metrics from an mTLS-enabled HTTP server.

If you are encrypting metrics for an application where you own the source code, then you can directly modify the code in your workload to ensure that your server supports mTLS and is configured with the mTLS certificate.

You might have cases where you cannot modify the source code of a application because you don't own it. In those cases, you must search for the configuration of the open source application that you are using to determine how to enable an HTTPS metrics endpoint. You can deploy the required configuration, for example, as command-line arguments or as a config file, depending on the application.

Enable mTLS metrics collection

To enable the collection of encrypted metrics, you must add the label monitoring.gdc.goog/enable-mtls="true" to your MonitoringTarget custom resource.

Use the following sample as a template:

  apiVersion: monitoring.gdc.goog/v1
  kind: MonitoringTarget
  metadata:
    name: "SERVICE_NAME"
    namespace: namespace
    labels:
      monitoring.gdc.goog/enable-mtls: "true" # Enable mTLS metrics collection
  spec:
    selector:
      matchLabels:
        app: "SERVICE_NAME"

To verify that you enabled mTLS authentication, check that your MonitoringTarget custom resource shows a Ready status for the certificate and the certificate secret. The MonitoringTarget custom resource must look like the following example:

apiVersion: monitoring.gdc.goog/v1
kind: MonitoringTarget
[...]
status:
  conditions:
  - lastTransitionTime: "2023-11-09T11:15:10Z"
    message: "admin,user-vm-1,user-vm-2,org-1-system"
    observedGeneration: 2
    reason: Ready
    status: "True"
    type: Ready
  - lastTransitionTime: "2023-11-09T11:14:43Z"
    message: "Certificate is ready"
    observedGeneration: 2
    reason: Ready
    status: "True"
    type: CertificateReady
  - lastTransitionTime: "2023-11-09T11:15:10Z"
    message: "Successfully created secret"
    observedGeneration: 2
    reason: Ready
    status: "True"
    type: CertificateSecretReady