Cortex Framework: integration with LiveRamp

This page describes the required configurations to bring data from LiveRamp as a data source of the marketing workload of Cortex Data Foundation.

LiveRamp is a data collaboration platform that helps businesses connect, control, and activate their data to improve customer experiences and drive better business outcomes. Cortex Framework offers the tools and platform to analyze it, combine it with other data sources, and use AI to gain deeper insights and optimize your marketing strategy. See Google Ads Reporting Data Model in the Cortex Data Foundation repository.

The following diagram describes how LiveRamp data source is available through the marketing workload of Cortex Data Foundation:

Configuration file

The config.json file configures the settings required to connect to data sources for transferring data from various workloads. This file contains the following parameters for LiveRamp:

   "marketing": {
        "deployLiveRamp": true,
        "LiveRamp": {
            "datasets": {
                "cdc": ""
            }
        }
    }

The following table describes the value for each LiveRamp parameter:

API connection

Cortex Data Foundation enables Identity Resolution by integrating with LiveRamp.Cortex Frameworks can perform RampID lookup, which allow businesses to identify known audiences or customer segments from their CRM systems.

RampID is an identifier generated by LiveRamp's API using Personally Identifiable Information (PII) like email, phone number, and name. This RampID allows businesses to identify and consolidate records across multiple systems, such as audiences of different campaigns. By integrating with LiveRamp, businesses can improve their ability to identify and target their audiences, leading to more effective and personalized marketing campaigns.

LiveRamp Identity Resolution Retrieval API allows businesses to programmatically resolve PII data to individuals. {product_name_short} uses LiveRamp's Lookup Endpoint by sending hashed PII data over the API call.

Input and output tables

After deploying Cortex Framework, the system creates the following two BigQuery tables in relevant dataset provided in config.json.

rampid_lookup_input table

The rampid_lookup_input table is the input for the RampID Lookup process.

The rampid_lookup_input table needs to be populated on a periodic basis (based on your business needs) with your customer's PII details like. Cortex Framework does not provide an automated way to do so, but it provides a sample script ddls/samples/populate_rampid_lookup_input.sql that shows how you could populate this table using data in your Salesforce system already deployed with Cortex Framework. You can use this file as a guide in case your data comes from another system.

Ensure you don't have duplicate entries in the rampid_lookup_input table (for example, the same individual is present multiple times with same PII information, even if their ID might be different). Cortex Framework Lookup DAG would fail if a segment contains many duplicate entries. This is enforced by LiveRamp APIs.

The rampid_lookup table

The rampid_lookup table is an output table containing RampIDs for each segment in the input record. LiveRamp by design does not allow to map RampID to an individual record.

LiveRamp RampIDs can change over a period of time, for the same individual. This means, you need to perform a fresh lookup for already processed data from time to time. Cortex Framework provides a sample script ddls/samples/clean_up_segment_matching.sql that shows how you can do this at a segment level. This way, you can reset a whole segment, and the system would perform a lookup for that segment and give you up to date RampIDs.

The rampid_lookup (output table) might contain slightly less number of records compared to the input table. This is by design, as Cortex Framework tries to debug the input records using PII details to ensure LiveRamp API lookup does not fail.

Account authentication

1. Contact LiveRamp to obtain authentication credentials. This should include Client ID and Client Secret.

2. Create a secret using Secret Manager with name cortex-framework-liveramp, and use the following as value. See instructions in Secret-Manager documentation.

   {
     'client_id':'CLIENT_ID',
     'client_secret':'CLIENT_SECRET',
     'grant_type':'client_credentials'
   }
   

   Replace the following:

   • 'CLIENT_ID' with the client ID obtained from step 1.
   • 'CLIENT_SECRET' with the client secret obtained from step 1.

Cloud Composer connections

Create the following connections in Cloud Composer. See more details in the Manage Airflow connections documentation.

Configuration

The config.ini file controls some behavior of the Cloud Composer DAG, as well as how LiveRamp APIs are consumed. Configure LiveRamp/src/pipelines/config.ini based on your needs. While these parameters are already described in the file, pay attention to liveramp_api_base_url. This parameter by default points to LiveRamp's production API URL, for testing purposes, you might need to point it to the staging version, depending on your setup.

What's next?

• For more information about other data sources and workloads, see Data sources and workloads.
• For more information about the steps for deployment in production environments, see Cortex Data Foundation deployment prerequisites.