Troubleshoot issues

This page shows you how to resolve issues that you might encounter when using Eventarc.

For destination-specific issues that you might also encounter, refer to troubleshooting for:

Eventarc provider

This section provides troubleshooting tips for issues at the source of events (the provider).

Provider doesn't exist

Check that the provider exists and that you have specified the trigger filters appropriately.

Events are produced from the provider, but are not received at the target

Check that you are filtering for events from providers that are in a supported region for the target destination. For more information, see locations supported by Eventarc and Understand Eventarc locations.

Trigger is not delivering events

  1. Verify that the provider is generating events. Check the Cloud Audit Logs and make sure the monitored service is writing logs. If logs are recorded but events are not delivered, contact support.

  2. Verify that a Pub/Sub topic with the same trigger name exists.

    1. To list triggers in all locations, run:

      gcloud eventarc triggers list --location=-

    2. To list Pub/Sub topics, run:

      gcloud pubsub topics list

      Verify that the Pub/Sub topic name includes the name of the created trigger. If the Pub/Sub topic is missing, create a topic when you create the trigger.

    3. To describe triggers and detect any issues with the condition of the trigger, run:

      gcloud eventarc triggers describe TRIGGER_ID \
  --location=LOCATION

      Replace the following:

      • TRIGGER_ID: the ID of the trigger or a fully qualified identifier.
      • LOCATION: the location of the Eventarc trigger.

    4. The previous command returns the condition of the trigger. If there is an issue with the Pub/Sub topic, one of the following conditions is returned:

      conditions:
transport.pubsub.topic:
  code: NOT_FOUND
  message: Pub/Sub topic not found. Try recreating the trigger.

      or

      conditions:
transport.pubsub.topic:
  code: UNKNOWN
  message: Pub/Sub topic status is unknown. Try requesting the trigger description again.

      If applicable, do the following:

      • If Pub/Sub topic not found is returned, recreate the trigger for a specific provider, event type, and destination](/eventarc/standard/docs/event-providers-targets#triggers) and then create the topic.

      • If Pub/Sub topic status is unknown is returned, run the command to describe the trigger again.

      • If the problem persists, contact support. For more information, see Manage triggers.

  3. Verify the health of the Pub/Sub topic:

    1. Verify the Pub/Sub topic configuration. Go to the Topics page.

      Go to Topics

    2. Monitor if messages are published to the topic with the metric: topic/send_message_operation_count. If messages are not published to the topic, check Cloud Audit Logs and make sure the monitored service is emitting logs. If logs are recorded but events are not delivered, contact support.

    3. Monitor if messages are pushed successfully with the metric: subscription/push_request_count by response_code. If push errors are reported, check the service logs. If the receiving endpoint returns a non-OK status code, it indicates that the Cloud Run code is not working as expected and you must contact support.

Eventarc trigger

This section provides troubleshooting tips for issues at the trigger.

Permission denied errors

  • If you encounter a Failed to load service details: permission denied error, confirm that the trigger permissions are configured correctly, and that the Pub/Sub service account is granted the iam.serviceAccountTokenCreator role. For more information, see Access control.

  • If you encounter the following error while trying to create a trigger, wait a few minutes (up to seven), and then try creating the trigger again:

    Permission denied while using the Eventarc Service Agent. If you recently
started to use Eventarc, it may take a few minutes before all necessary
permissions are propagated to the Service Agent. Otherwise, verify that it has
Eventarc Service Agent role.

    A service agent acts as the identity of a given Google Cloud service for a particular project. For more information, see Service agents and view the permissions for the Eventarc service agent role (roles/eventarc.serviceAgent).

    To verify that the Eventarc service agent exists in your Google Cloud project and has the necessary role, complete the following steps:

    1. In the Google Cloud console, go to the IAM page.

      Go to IAM

    2. On the Permissions tab, select the Include Google-provided role grants checkbox.

    3. In the list of principals, locate the Eventarc service agent, which uses this format:

      service-PROJECT_NUMBER@gcp-sa-eventarc.iam.gserviceaccount.com

    4. Verify that the service agent has the Eventarc Service Agent role. If the service agent does not have the role, grant the role.

  • If you are trying to use a cross-project service account and you encounter a permission denied error for iam.serviceAccounts.getAccessToken or iam.serviceAccounts.actAs, or if you receive an error message similar to ...cross-project service accounts are disabled or Failed to impersonate..., confirm that permissions are configured correctly. For more information, see Use a cross-project service account.

Pub/Sub topic not displaying CMEK key

Enabling an Eventarc channel with a customer-managed encryption key (CMEK) protects the Pub/Sub topic used as a transport layer by Eventarc Standard. However, the CMEK key is not displayed through the Google Cloud console. You can verify that a CMEK key is enabled for the Pub/Sub topic by using the Google Cloud CLI:

  1. Describe the Eventarc trigger:

      gcloud eventarc triggers describe TRIGGER_ID \
      --location=LOCATION

    Replace the following:

    • TRIGGER_ID: the ID of your Eventarc trigger

    • LOCATION: the location of the trigger

      The Pub/Sub topic ID is returned. For example: topic: projects/PROJECT_ID/topics/TOPIC_ID

  2. Describe the Pub/Sub topic:

      gcloud pubsub topics describe TOPIC_ID

    Replace TOPIC_ID with the Pub/Sub topic ID retrieved in the previous step.

    The CMEK key name is returned. For example:

      kmsKeyName: projects/PROJECT_ID/locations/LOCATION/keyRings/RING_NAME/cryptoKeys/KEY_NAME/grants/GRANT_ID

Unexpected charges when using Eventarc

When you create a trigger, check if the filters configured could lead to triggers being set off indefinitely. To help avoid unexpected charges on your bill, set budgets and alerts using Google Cloud's billing console. For more information, see Create a budget.