Configure notification channels for AO alerts

The Observability platform lets you configure notification channels for the alerts you set up. The Observability system sends an alert to these customized channels whenever an event causes an alert. This way, you can notify the recipients you need according to your preferred methods.

Google Distributed Cloud (GDC) air-gapped does not predefine notification channels intended for sending alerts. You must define at least one channel to receive alerts. If you don't define a notification channel configuration, the GDC Observability system takes the configuration from the platform resources as a default. Consequently, if you don't configure notification channels but an administrator did it for the organization, then the system sends alerts to those channels.

Before you begin

To get the permissions you need to configure or view notification channels, ask your Project IAM Admin to grant you one of the following roles in your project namespace:

Observability Pipeline Editor: edits or modifies ObservabilityPipeline custom resources. Request the Observability Pipeline Editor (observabilitypipeline-editor) role.
Observability Pipeline Viewer: views ObservabilityPipeline custom resources. Request the Observability Pipeline Viewer (observabilitypipeline-viewer) role.

Additionally, to get the permissions you need to create ConfigMap objects in the project namespace, ask your Project IAM Admin to grant you the ConfigMap Creator (configmap-creator) role.

For more information about roles and access, see Grant access to project resources.

Configure notification channels

Work through the following steps to configure notification channels for the alerts you set up for system monitoring:

Define the Observability alerting configuration following the specification in https://prometheus.io/docs/alerting/latest/configuration/.
Create a ConfigMap object and deploy it into your project namespace. The object must contain the configuration rules in the alertmanager.yml file.
```
apiVersion: v1
kind: ConfigMap
metadata:
  # The name must match the ConfigMap name specified in the ObservabilityPipeline custom resource.
  name: CUSTOMIZED_CONFIGMAP_NAME
  # The namespace must match the project namespace
  namespace: PROJECT_NAMESPACE
data:
  # The file name must be alertmanager.yml
  alertmanager.yml: |
    # Add the customized configuration in the following lines of this file.
```
Replace the following:
- CUSTOMIZED_CONFIGMAP_NAME: the name of your configuration file. You use this name in the ObservabilityPipeline custom resource during the following step.
- PROJECT_NAMESPACE: the namespace of your project
Note: Follow the same syntax as described in https://prometheus.io/docs/alerting/latest/configuration to define the notification channels in the alertmanager.yml file.

Configure the ObservabilityPipeline custom resource in a command-line editor to import the configuration to the alertmanagerConfig field.

 # Configure observability pipeline
apiVersion: observability.gdc.goog/v1
kind: ObservabilityPipeline
metadata:
  # The namespace must match the project namespace
  namespace: PROJECT_NAMESPACE
  name: observability-config
spec:
  # Configure alerts
  alerting:
    # The alerting configuration must be in the key "alertmanager.yml" file of the configMap object
    alertmanagerConfig: CUSTOMIZED_CONFIGMAP_NAME
  ...

Replace the following:

PROJECT_NAMESPACE: the namespace of your project
CUSTOMIZED_CONFIGMAP_NAME: the name you used for your configuration file in the ConfigMap object. The name must match exactly.

Save the changes in the ObservabilityPipeline custom resource and exit your command-line editor.