Subscribe to events from Datadog

Standard

Datadog supports sending events to Eventarc for delivery to interested subscribers.

Datadog is an observability service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform. Learn more about using Datadog.

To subscribe to events from Datadog, complete the following steps:

Before you begin

Ensure that you have enabled the Eventarc and Eventarc Publishing APIs:
Console
Enable the APIs
gcloud
```
gcloud services enable \
    eventarc.googleapis.com \
    eventarcpublishing.googleapis.com
```
If you are enabling a customer-managed encryption key (CMEK) for a channel, enable the Cloud Key Management Service API.
Console
Enable the API
gcloud
```
gcloud services enable cloudkms.googleapis.com
```

Retrieve the provider name used by Datadog

The provider name is required when creating and using a channel. You can list all Eventarc providers and retrieve their details, including supported event types, by running gcloud commands.

For example, you can list all third-party providers in any location:

gcloud eventarc providers list --filter='eventTypes.type!~^google*' --location=-

The output includes something similar to the following:

NAME: datadog
LOCATION: us-central1

For details, see List event providers.

Retrieve the events supported by Datadog

An event type is needed when creating an Eventarc trigger. You can list Datadog's supported event types, by running a gcloud command:

gcloud eventarc providers describe datadog \
    --location=LOCATION

Replace LOCATION with a supported Eventarc region.

For example:

gcloud eventarc providers describe datadog --location=us-central1

The output is similar to the following:

displayName: Datadog
eventTypes:
- type: datadog.v1.alert
name: projects/project-id/locations/us-central1/providers/datadog
[...]

Make note of the event type to use when creating an Eventarc trigger.

Create a channel for Datadog events

To subscribe to events from Datadog, you must create a channel:

Console

In the Google Cloud console, go to the Eventarc > Channels page.

Go to Channels
Click Create channel.
Select a Channel type and Provider.

For example, select Third-party > Datadog.
In the Channel name field, enter a name for the channel.
In the Region list, select a supported Eventarc region.
Optionally, select the Use a customer-managed encryption key (CMEK) checkbox and, in the CMEK encryption key list, select a key ring that you have created for the region.

For more information, see Use customer-managed encryption keys.
Click Create.

gcloud

gcloud eventarc channels create CHANNEL_NAME \
    --provider datadog \
    --project PROJECT_ID \
    --location LOCATION

Replace the following:

CHANNEL_NAME: a name for the channel
PROJECT_ID: your Google Cloud project ID

You can find your project ID on the Welcome page of the Google Cloud console.
LOCATION: a supported Eventarc region

Note: For this Pre-GA feature, only us-central1 is supported as a location. You cannot create a channel in any other location.

You can optionally use a --crypto-key flag to specify a fully qualified Cloud KMS key name in the format projects/PROJECT_NAME/locations/LOCATION/keyRings/RING_NAME/cryptoKeys/KEY_NAME.

For more information, see Use customer-managed encryption keys.

For details, see Create a channel.

Retrieve specific details to convey to Datadog

After creating a channel, you can retrieve specific details to convey to Datadog:

Console

In the Google Cloud console, go to the Eventarc > Channels page.

Go to Channels
From the list of channels, click a channel to view its details.

The channel status should be Pending.
Copy the Channel full name and Activation token value. Both will be needed later to convey to Datadog and activate the channel.

gcloud

gcloud eventarc channels describe CHANNEL_NAME \
    --project PROJECT_ID \
    --location LOCATION

The output is similar to the following:

activationToken: aS7dXs1b79AcXsf
createTime: '2021-11-15T15:20:31.582356065Z'
name: projects/project-id/locations/us-central1/channels/datadog-channel
provider: projects/project-id/locations/us-central1/providers/datadog
pubsubTopic: projects/project-id/topics/eventarc-channel-us-central1-datadog-channel-077
state: PENDING
uid: c6703a91-ccd3-4c32-a729-967393f23a29
updateTime: '2021-11-15T15:21:03.689597653Z'

Note that the channel is in a PENDING state. To activate the channel, you must provide the activationToken value and the channel name to Datadog.

Learn more about the channel state.

Send the channel details to Datadog

Before you can start consuming events, Datadog must initialize the connection between their ecosystem and the channel.

Log into Datadog at https://www.datadoghq.com.
In the navigation menu, select Integrations.
From the list of integrations, select Google Eventarc.
A green checkmark confirms the installation:
Click Configuration.
In the Add New Channels section, click Add New.
Enter the following channel details:
- Channel full name: the channel name, including the Google Cloud project ID and location of the resource.
  For example:
```
projects/PROJECT_ID/locations/LOCATION/channels/CHANNEL_NAME
```
- Activation token: the token is valid for 24 hours after the channel's creation. After 24 hours, the channel becomes INACTIVE.
  For example: aS7dXs1b79AcXsf
To save the configuration, click the checkmark .
The new channel is now listed.

Once Datadog receives the required information, and has initialized the connection between their ecosystem and the channel, the state of the channel changes from PENDING to ACTIVE. Datadog can now send events to your project and you can start consuming the events.

Deploy an event receiver

To test the configuration, deploy an event receiver—in this case, a Workflows workflow that logs received events—and create an Eventarc trigger that routes events from Datadog to the event receiver. You must use a service account that has been granted the appropriate roles to access your Google Cloud project resources.

Enable the Workflows and Workflow Executions APIs:

Console

Enable the APIs

gcloud

gcloud services enable \
    workflows.googleapis.com \
    workflowexecutions.googleapis.com

Create a service account and grant it the roles/logging.logWriter and roles/workflows.invoker roles:
Console
1. In the Google Cloud console, go to the Service Accounts page.
  
  Go to Service Accounts
2. Select your project.
3. In the Service account name field, enter a name. The Google Cloud console fills in the Service account ID field based on this name. Optionally, add a description.
4. Click Create and continue.
5. In the Select a role list, select Logging > Logs Writer.
6. Click Add another role and select Workflows > Workflows Invoker.
7. Click Continue.
8. To finish creating the service account, click Done.
gcloud
```
gcloud iam service-accounts create MY_SERVICE_ACCOUNT
gcloud projects add-iam-policy-binding PROJECT_ID \
    --role="roles/workflows.invoker" \
    --member="serviceAccount:MY_SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding PROJECT_ID \
    --role="roles/logging.logWriter" \
    --member="serviceAccount:MY_SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
```
Replace MY_SERVICE_ACCOUNT with a name for your service account.
For more information about how to control access to Workflows resources, see Use IAM to control access.
Create and deploy a workflow that logs a received event:
Console
1. In the Google Cloud console, go to the Workflows page.
  
  Go to Workflows
2. Click Create.
3. Name your workflow logEventsWorkflow.
4. Choose a supported Workflows region; for example, us-central1.
5. Select the Service account you previously created.
6. Click Next.
7. In the workflow editor, enter the following definition for your workflow:
```
main:
  params: [event]
  steps:
  - logStep:
      call: sys.log
      args:
        data: ${event}
```
8. Click Deploy.
gcloud
1. Open a terminal or Cloud Shell.
2. In your home directory, create a new file called logEventsWorkflow.yaml.
3. Copy and paste the following into the new file and save it:
```
main:
  params: [event]
  steps:
  - logStep:
      call: sys.log
      args:
        data: ${event}
```
4. Deploy the workflow:
```
gcloud workflows deploy logEventsWorkflow \
    --source=logEventsWorkflow.yaml \
    --location LOCATION
```
  Replace LOCATION with a supported Workflows region; for example, us-central1.
Create an Eventarc trigger to route events from Datadog to the target workflow:
Console
1. In the Google Cloud console, go to the Eventarc > Triggers page.
  
  Go to Triggers
2. Click Create trigger.
3. Type a Trigger name.
  
  This is the ID of the trigger and it must start with a letter. For example, my-event-channel-trigger.
4. Select a Trigger type and Event provider.
  
  This is the partner service that is the source of events. For example, select Third-party > Datadog.
5. Select the appropriate Channel.
6. Select an Event.
  
  For example: datadog.v1.alert
7. Optionally, in the Filters section, specify an appropriate attribute and value.
8. Select the Service account you previously created to invoke your workflow.
9. For the Event destination, select Workflows, and then select the workflow you previously created.
10. Click Create.
gcloud
```
gcloud eventarc triggers create TRIGGER \
    --location=LOCATION \
    --destination-workflow=logEventsWorkflow  \
    --event-filters=type=datadog.v1.alert \
    --channel=CHANNEL_NAME \
    --project=PROJECT_ID \
    --service-account="MY_SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
```
Replace TRIGGER with the ID of the trigger or a fully qualified identifier. For example, my-event-channel-trigger.
For more information, see Create a trigger for a third-party provider.

Test the routing of events

You can now test the entire configuration and ensure that an event from Datadog is routed to the workflow event receiver which logs the received event.

To create a Datadog monitor, log into Datadog:
1. In the navigation menu, select Monitors > New Monitor.
2. From the list of Custom monitor types, select Metric.
3. Accept the following default settings:
  1. Choose the detection method
  2. Define the metric
4. Set alert conditions: for the Alert threshold, enter 1.
5. Notify your team: give your monitor a name and, from the Notify your services and your team members list, select the appropriate value. The message should be similar to the following:
  
  @eventarc-PROJECT_ID_LOCATION_CHANNEL_NAME
6. Define permissions and audit notifications: select the appropriate values.
7. Click Test Notifications.
8. In the Test notifications for this monitor dialog, click Run Test.
  
  This triggers an execution of the workflow.
For more information about configuring a Datadog monitor, see the Datadog documentation.

To verify that a workflow execution was triggered, list the last five executions:

gcloud workflows executions list ${MY_WORKFLOW} --limit=5

The output should be similar to the following, listing a NAME and STATE equal to SUCCEEDED for each workflow execution.

NAME: projects/1051295516635/locations/us-central1/workflows/logEventsWorkflow/executions/674b5783-deec-4d1b-be1d-b067f7b32971
STATE: SUCCEEDED
START_TIME: 2022-07-13T22:01:06.314980819Z
END_TIME: 2022-07-13T22:01:06.637825944Z

Every time a workflow executes a single run of the logic contained in a workflow's definition, it automatically triggers at least two execution logs: one at the start of an execution and one at the end. To view the logs do the following:

In the Google Cloud console, go to the Workflows page:
Go to Workflows
To access a workflow's Details page, click the workflow's name.
To view the logs, click Logs. The output should be similar to the following:

{
  "alert_type": "error",
  "data": {
    "message": "@eventarc-test-project_us-central1_datadog-channel\nTest notification"
  },
  "datacontenttype": "application/json",
  "id": 6601612988428281000,
  "monitor": "monitor",
  "monitor_id": "1234567",
  "msg_title": "Triggered on 'TEST' Test monitor for Eventarc",
  "priority": "None",
  "source": "//datadog/alert",
  "specversion": "1.0",
  "time": "2022-07-13T22:01:03Z",
  "type": "datadog.v1.alert"
}

What's next

Get started using events from Datadog through the codelab
Learn more about third-party events
Learn more about Eventarc