Cortex Framework: integration with Google Ads

This page describes the required configurations to bring data from Google Ads as a data source of the marketing workload of Cortex Data Foundation.

Google Ads is an online advertising platform that allows businesses to advertise their products or services across various Google properties. Cortex Framework brings your Google Ads data together with other marketing channels, analyzes it comprehensively, and uses AI to improve your campaign results. See Google Ads Reporting Data Model in the Cortex Data Foundation repository.

The following diagram describes how Google Ads data is available through the marketing workload of Cortex Data Foundation:

Configuration file

The config.json file configures the settings required to transfer data from any data source, including Google Ads. This file contains the following parameters for Google Ads:

  "marketing": {
          "deployGoogleAds": true,
          "GoogleAds": {
              "deployCDC": true,
              "lookbackDays": 180,
              "datasets": {
                  "cdc": "",
                  "raw": "",
                  "reporting": "REPORTING_GoogleAds"
                    }
                  }
                 }

The following table describes the value for each Google Ads marketing parameter:

API connection

Cortex Framework ingestion templates use the Google Ads API to retrieve reporting attributes and metrics from Google Ads. The current Cortex Framework templates use Google Ads API version 15. Consider the Google Ads API limitations:

• Basic access operations per day: 15000 (paginated requests containing valid next_page_token are not counted).
• Max page size: 10000 rows per page.
• Recommended default parameters: Page size equals to 10000 rows per page.

For more information about the API connection, see Google Ads API documentation..

Account authentication

Follow these steps to set up account authentication:

1. In the Google Cloud console, click Navigation menu > API & Services > Credentials > Create credentials.

2. Create a OAuth Client ID credential with the following characteristics. For more information, see Using OAuth 2.0 to Access Google APIs.

   Application type: "Web Application"
   Name: CHOSEN_NAME #(For example,"Cortex Authentication Client").
   Authorized redirect URIs: http://127.0.0.1
   

   Replace CHOSEN_NAME with the chosen name for OAuth Client ID credential account.

3. Save the Client ID and Client secret after the credential is configured. You need it later.

4. Generate a fresh token using OAuth 2.0 Access Google APIs. Cortex Data Foundation automatically detects and ingest data from all customers (accounts) that are accessible to the credentials used to generate the token.

5. Create a secret using Secret Manager:

   • In the Google Cloud console, click Secret Manager.
   • Create a secret called cortex-framework-google-ads-yaml using the following format and changing the values according with your settings:

   {"developer_token": "DEVELOPER_TOKEN_VALUE", "refresh_token": "REFRESH_TOKEN_VALUE", "client_id": "CLIENT_ID_VALUE", "client_secret": "CLIENT_SECRET_VALUE", "use_proto_plus": False}
   

Replace the following:

• DEVELOPER_TOKEN_VALUE with the developer token value available in Google Ads account.
• REFRESH_TOKEN_VALUE with the refresh token value obtained in step 4.
• CLIENT_ID_VALUE with the client ID value obtained in the OAuth setup in step 2.
• CLIENT_SECRET_VALUE with the client secret value obtained from the OAuth setup in step 2.

Cloud Composer connections permissions

Create the following connections in Cloud Composer. See more details in the Manage Airflow connections documentation.

Cloud Composer service account permissions

Grant Dataflow permissions to the service account used in Cloud Composer (as configured in the googleads_raw_dataflow connection). See instructions in Dataflow documentation.

Ingestion settings

Control Source to Raw and Raw to CDC data pipelines through the settings in the file src/GoogleAds/config/ingestion_settings.yaml. This section describes the parameters of each data pipeline.

Source to raw tables

This section describes which entities are fetched by APIs and how. Each entry corresponds with one Google Ads entity. Based on this config, Cortex creates Airflow DAGs that run Dataflow pipelines to fetch data using Google Ads APIs.

The following parameters control the settings for Source to Raw for each entry:

Raw to CDC tables

This section describes which entries control how data is moved from raw tables to CDC tables. Each entry corresponds with a raw table (which in turn corresponds with Google Ads API entity as mentioned).

The following parameters control the settings for Raw to CDC for each entry:

Reporting settings

You can configure and control how Cortex Framework generates data for the Google Ads final reporting layer using the reporting settings file src/GoogleAds/config/reporting_settings.yaml. This file controls how reporting layer BigQuery objects (tables, views,functions or stored procedures) are generated.

For more information, see Customizing reporting settings file.

What's next?

• For more information about other data sources and workloads, see Data sources and workloads.
• For more information about the steps for deployment in production environments, see Cortex Data Foundation deployment prerequisites.