Receive a Cloud Audit Logs event

This tutorial shows you how to deploy an authenticated Cloud Run service that receives events from Cloud Storage using Cloud Audit Logs. Use this tutorial to deploy production workloads. The Eventarc trigger filters events based on Cloud Audit Logs entries. For more information, see Determine event filters for Cloud Audit Logs.

You can complete this tutorial using either the Google Cloud console or the Google Cloud CLI.

Objectives

In this tutorial, you will:

  1. Create a Cloud Storage bucket to be the event source.

  2. Deploy an event receiver service to Cloud Run.

  3. Create an Eventarc trigger.

  4. Generate an event by uploading a file to the Cloud Storage bucket, and view it in the Cloud Run logs.

Costs

In this document, you use the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

Before you begin

Security constraints defined by your organization might prevent you from completing the following steps. For troubleshooting information, see Develop applications in a constrained Google Cloud environment.

If you are the project creator, you are granted the basic Owner role (roles/owner). By default, this Identity and Access Management (IAM) role includes the permissions necessary for full access to most Google Cloud resources and you can skip this step.

If you are not the project creator, required permissions must be granted on the project to the appropriate principal. For example, a principal can be a Google Account (for end users) or a service account (for applications and compute workloads). For more information, see the Roles and permissions page for your event destination.

Note that by default, Cloud Build permissions include permissions to upload and download Artifact Registry artifacts.

Required permissions

To get the permissions that you need to complete this tutorial, ask your administrator to grant you the following IAM roles on your project:

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Complete the following steps using the Google Cloud console or the gcloud CLI:

Console

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. Enable the Artifact Registry, Cloud Build, Logging, Pub/Sub, Cloud Run, Cloud Storage, and Eventarc APIs.

    Enable the APIs

    5.

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  6. Make sure that billing is enabled for your Google Cloud project.

  7. Enable the Artifact Registry, Cloud Build, Logging, Pub/Sub, Cloud Run, Cloud Storage, and Eventarc APIs.

    Enable the APIs

    8.
  8. To configure what data access is recorded in your audit logs, enable the Admin Read, Data Read, and Data Write log types for Google Cloud Storage:

    Go to Audit Logs

  9. In the Google Cloud console, go to the Service accounts page.

    Go to Service Accounts

  10. Click Create service account.

  11. Enter a service account name to display in the Google Cloud console.

    The Google Cloud console generates a service account ID based on this name. Edit the ID if necessary. You cannot change the ID later.

  12. Optional: Enter a description of the service account.

  13. If you don't want to set access controls now, click Done to finish creating the service account.

    To set access controls now, click Create and continue and continue to the next step.

  14. Select the Cloud Run Invoker and Eventarc Event Receiver roles to grant on the project to the service account associated with your Eventarc trigger.

    For testing purposes, this grants the Cloud Run Invoker role on all Cloud Run services and jobs in the project; however, you can grant the role on the service. For more information, see Grant Cloud Run service permissions.

    Note that if you create a trigger for an authenticated Cloud Run service without granting the Cloud Run Invoker role, the trigger is created successfully and is active. However, the trigger won't work as expected and a message similar to the following appears in the logs:

    The request was not authenticated. Either allow unauthenticated invocations or set the proper Authorization header.
  15. When you are done adding roles, click Continue and Done to finish creating the service account.

gcloud

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. Install the Google Cloud CLI.

  3. To initialize the gcloud CLI, run the following command: 

    gcloud init

  4. Create or select a Google Cloud project.

    • Create a Google Cloud project:

      gcloud projects create PROJECT_ID

      Replace PROJECT_ID with a name for the Google Cloud project you are creating.

    • Select the Google Cloud project that you created:

      gcloud config set project PROJECT_ID

      Replace PROJECT_ID with your Google Cloud project name.

  5. Make sure that billing is enabled for your Google Cloud project.

  6. Enable the Artifact Registry, Cloud Build, Logging, Pub/Sub, Cloud Run, Cloud Storage, and Eventarc APIs: 

    gcloud services enable artifactregistry.googleapis.com cloudbuild.googleapis.com 
      logging.googleapis.com  pubsub.googleapis.com  run.googleapis.com 
      storage.googleapis.com  and eventarc.googleapis.com
  7. Install the Google Cloud CLI.

  8. To initialize the gcloud CLI, run the following command: 

    gcloud init

  9. Create or select a Google Cloud project.

    • Create a Google Cloud project:

      gcloud projects create PROJECT_ID

      Replace PROJECT_ID with a name for the Google Cloud project you are creating.

    • Select the Google Cloud project that you created:

      gcloud config set project PROJECT_ID

      Replace PROJECT_ID with your Google Cloud project name.

  10. Make sure that billing is enabled for your Google Cloud project.

  11. Enable the Artifact Registry, Cloud Build, Logging, Pub/Sub, Cloud Run, Cloud Storage, and Eventarc APIs: 

    gcloud services enable artifactregistry.googleapis.com cloudbuild.googleapis.com 
      logging.googleapis.com  pubsub.googleapis.com  run.googleapis.com 
      storage.googleapis.com  and eventarc.googleapis.com
  12. Update gcloud components: 
    gcloud components update
  13. Sign in using your account: 
    gcloud auth login
  14. Set the configuration variables used in this tutorial: 
    export REGION=us-central1
gcloud config set run/region ${REGION}
gcloud config set run/platform managed
gcloud config set eventarc/location ${REGION}
export SERVICE_NAME=helloworld-events
  15. To configure what data access is recorded in your audit logs, enable the ADMIN_READ, DATA_READ, and DATA_WRITE log types for the storage.googleapis.com service:
    1. Read your project's IAM policy and store it in a file: 
      gcloud projects get-iam-policy PROJECT_ID > /tmp/policy.yaml
    2. Edit your policy in /tmp/policy.yaml, adding or changing only the Data Access audit logs configuration. 
      auditConfigs:
- auditLogConfigs:
- logType: ADMIN_READ
- logType: DATA_WRITE
- logType: DATA_READ
service: storage.googleapis.com
    3. Write your new IAM policy: 
      gcloud projects set-iam-policy PROJECT_ID /tmp/policy.yaml
       If the preceding command reports a conflict with another change, then repeat these steps, starting with reading the project's IAM policy.
  16. Create a service account for the project: 
    gcloud iam service-accounts create sample-service-account \
    --description="A sample service account" \
    --display-name="Sample service account"
     After you create a service account, it can take up to seven minutes before you can use the service account. If you try to use a service account immediately after you create it, and you receive an error, wait at least 60 seconds and try again.
  17. To confirm that sample-service-account has been created, run: 
    gcloud iam service-accounts list
     The output should be similar to the following: 
    DISPLAY NAME                     EMAIL                                                               DISABLED
Default compute service account  PROJECT_NUMBER-compute@developer.gserviceaccount.com                False
Sample service account           sample-service-account@PROJECT_ID.iam.gserviceaccount.com           False
  18. Grant the Cloud Run Invoker (run.invoker) and the Eventarc Event Receiver (roles/eventarc.eventReceiver) roles on the project to the service account associated with your Eventarc trigger: 
    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="serviceAccount:sample-service-account@PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/run.invoker"
     
    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="serviceAccount:sample-service-account@PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/eventarc.eventReceiver"

    Replace PROJECT_ID with your Google Cloud project ID.

    For testing purposes, this grants the Cloud Run Invoker role on all Cloud Run services and jobs in the project; however, you can grant the role on the service. For more information, see Grant Cloud Run service permissions.

    Note that if you create a trigger for an authenticated Cloud Run service without granting the Cloud Run Invoker role, the trigger is created successfully and is active. However, the trigger won't work as expected and a message similar to the following appears in the logs:

    The request was not authenticated. Either allow unauthenticated invocations or set the proper Authorization header.

Create an Artifact Registry standard repository

Create an Artifact Registry standard repository to store your container image.

Console

  1. In the Google Cloud console, go to the Repositories page.

    Go to Repositories

  2. Click Create repository.
  3. Configure your repository:
    1. Enter a unique Name.
    2. For Format, choose Docker.
    3. For Mode, choose Standard.
    4. For Location type, choose Region.
    5. In the Region list, select us-central1 (Iowa).
  4. Accept the other defaults and click Create.

gcloud

gcloud artifacts repositories create REPOSITORY \
    --repository-format=docker \
    --location=$REGION

Replace REPOSITORY with a unique name for the Artifact Registry repository.

Create a Cloud Storage bucket

This tutorial uses Cloud Storage as the event source. To create a storage bucket:

Console

  1. In the Google Cloud console, go to the Buckets page.

    Go to Buckets

  2. Click Create bucket.
  3. Enter your bucket information and click Continue to complete each step:
    1. Enter a unique Name. For example, eventarcbucket.
    2. Select Region as the Location Type.
    3. Select us-central1 (Iowa) as the Location.
    4. Select Standard for default storage class.
    5. Select Uniform for Access control.
  4. Click Create.

gcloud

gsutil mb -l $REGION gs://events-tutorial-PROJECT_ID/

After the event source is created, you can deploy the event receiver service on Cloud Run.

Deploy the event receiver service to Cloud Run

Deploy a Cloud Run service that receives and logs events. To deploy the sample event receiver service:

Console

  1. Clone the sample repository to your GitHub account:

    2. Go

    1. On GitHub, navigate to GoogleCloudPlatform/golang-samples
    2. Click Fork.
      3. Screenshot of the Fork button.
    3. If prompted, select the location where you want to fork the repository.

    Java

    1. On GitHub, navigate to GoogleCloudPlatform/java-docs-samples.
    2. Click Fork.
      3. Screenshot of the Fork button.
    3. If prompted, select the location where you want to fork the repository.

    .NET

    1. On GitHub, navigate to GoogleCloudPlatform/dotnet-docs-samples.
    2. Click Fork.
      3. Screenshot of the Fork button.
    3. If prompted, select the location where you want to fork the repository.

    Node.js

    1. On GitHub, navigate to GoogleCloudPlatform/nodejs-docs-samples.
    2. Click Fork.
      3. Screenshot of the Fork button.
    3. If prompted, select the location where you want to fork the repository.

    Python

    1. On GitHub, navigate to GoogleCloudPlatform/python-docs-samples.
    2. Click Fork.
      3. Screenshot of the Fork button.
    3. If prompted, select the location where you want to fork the repository.
  2. In the Google Cloud console, go to the Services page.

    3. Go to Services

  3. Click Create service to display the Create service form.
  4. Select Continuously deploy from a repository.

    Changes to your GitHub repository are automatically built into container images in Artifact Registry and deployed to Cloud Run.

  5. Click Set up with Cloud Build to open the Set up with Cloud Build form.
    1. If prompted, enable the Cloud Build API and Artifact Analysis API.
    2. Select GitHub as the Repository Provider.
    3. If prompted, click Install Google Cloud Build.
    4. Select the GitHub repository you forked as the Repository.
    5. Click Next.
    6. In the Branch field, enter ^main$.
    7. Select Dockerfile as the Build Type and provide the source location of the Dockerfile:
      • eventarc/audit-storage/Dockerfile

        or

      • eventarc/audit_storage/Dockerfile(Go)
    8. Click Save.
  6. In the Create service form, enter a service name. For example, helloworld-events.
  7. Select us-central1(Iowa) as the Region where you want your service located.
  8. Select any of the Ingress options based on the ingress traffic you want to allow on the Cloud Run service.
  9. Select Require authentication
  10. Click Create.

gcloud

  1. Clone the GitHub repository:

    Go

    git clone https://github.com/GoogleCloudPlatform/golang-samples.git
cd golang-samples/eventarc/audit_storage

    Java

    git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git
cd java-docs-samples/eventarc/audit-storage

    .NET

    git clone https://github.com/GoogleCloudPlatform/dotnet-docs-samples.git
cd dotnet-docs-samples/eventarc/audit-storage

    Node.js

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git
cd nodejs-docs-samples/eventarc/audit-storage

    Python

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
cd python-docs-samples/eventarc/audit-storage
  2. Build the container and upload it to Cloud Build: 
    gcloud builds submit --tag $REGION-docker.pkg.dev/PROJECT_ID/REPOSITORY/$SERVICE_NAME:v1
  3. Deploy the container image to Cloud Run: 
    gcloud run deploy $SERVICE_NAME \
    --image $REGION-docker.pkg.dev/PROJECT_ID/REPOSITORY/$SERVICE_NAME:v1

    At the prompt to allow unauthenticated invocations, enter n.

    When the deployment succeeds, the command line displays the service URL.

Now that you have deployed your event receiver service called helloworld-events to Cloud Run, you can set up your trigger.

Create an Eventarc trigger

The Eventarc trigger will send events from the Cloud Storage bucket to the helloworld-events Cloud Run service.

Console

  1. In the Google Cloud console, go to the Triggers page.

    Go to Triggers

  2. Click Create trigger.
  3. Type a Trigger name.

    This is the ID of the trigger and it must start with a letter. It can contain up to 63 lowercase letters, numbers, or hyphens.

  4. Select a Trigger type:
    • First-party: Filters events sent from Google Cloud providers (directly or through Cloud Audit Logs entries) or providers using Pub/Sub messages.
    • Third-party: Filters events sent from third-party providers.
  5. In the Event provider list, select Cloud Storage as the source of events.
  6. In the Event type list, select storage.objects.create.
  7. In the Region list, select us-central1 as the region to receive events from.
  8. Select the Service account you created. For example, SERVICE_ACCOUNT_ID@PROJECT_ID.iam.gserviceaccount.com.
  9. In the Event destination list, select Cloud Run.
  10. In the Select a Cloud Run service list, select helloworld-events.
  11. Click Create.
    12. Note that you can also create an Eventarc trigger using the Cloud Run console page.

gcloud

  1. Create a trigger that filters Cloud Storage events and that uses the service account you created: 
    gcloud eventarc triggers create events-tutorial-trigger \
    --destination-run-service=$SERVICE_NAME \
    --destination-run-region=$REGION \
    --event-filters="type=google.cloud.audit.log.v1.written" \
    --event-filters="serviceName=storage.googleapis.com" \
    --event-filters="methodName=storage.objects.create" \
    --service-account=sample-service-account@PROJECT_ID.iam.gserviceaccount.com
     Replace PROJECT_ID with the Google Cloud project ID.

    Where:

    • type: specifies that audit logs are created when the trigger's filter criteria is met.
    • serviceName: the service that writes the audit log, which is Cloud Storage.
    • methodName: the operation that is being audited, which is storage.objects.create.
    This creates a trigger called events-tutorial-trigger.
  2. To confirm events-tutorial-trigger was successfully created, run: 
    gcloud eventarc triggers list --location=$REGION
    3. The trigger, events-tutorial-trigger, is listed with a target of helloworld-events.

Generate and view an event

  1. To generate an event:

    Console

    1. Create a text file with the filename random.txt and the text "Hello World".
    2. In the Google Cloud console, go to the Buckets page.

      Go to Buckets

    3. Select the storage bucket you created.
    4. In the Objects tab, click Upload files and upload the random.txt file.

    gcloud

    Upload a text file to Cloud Storage:

    echo "Hello World" > random.txt
gsutil cp random.txt gs://events-tutorial-PROJECT_ID/random.txt
  2. To view the log entry:

    3. Console

    1. In the Google Cloud console, go to the Services page.

      Go to Services

    2. From the list of services, click the name of the service you created to go to its Service details page.
    3. Click the Logs tab, to get the request and container logs for all revisions of this service. You can filter by log severity level.
    4. Look for a log entry similar to: 
      Detected change in Cloud Storage bucket: storage.googleapis.com/projects/_/buckets/BUCKET_NAME/objects/random.txt
       where BUCKET_NAME is the name of the Cloud Storage bucket.

    gcloud

    1. gcloud logging read "resource.labels.service_name=helloworld-events AND textPayload:random.txt" --format=json
    2. Look for a log entry similar to: 
      Detected change in Cloud Storage bucket: storage.googleapis.com/projects/_/buckets/BUCKET_NAME/objects/random.txt
       where BUCKET_NAME is the name of the Cloud Storage bucket.

You have successfully deployed an event receiver service to Cloud Run, created an Eventarc trigger, generated an event from Cloud Storage, and viewed it in the Cloud Run logs.

Clean up

While Cloud Run does not charge when the service is not in use, you might still be charged for storing the container image in Artifact Registry, storing files in your Cloud Storage bucket, and Eventarc resources.

You can:

  1. Delete your container image.

  2. Delete your storage bucket.

  3. Delete the Eventarc trigger.

Alternatively, you can delete your Google Cloud project to avoid incurring charges. Deleting your Google Cloud project stops billing for all the resources used within that project.

Delete a Google Cloud project:

gcloud projects delete PROJECT_ID

What's next