Feed management API

This reference guide explains how to use the Google Security Operations feed management API to programmatically create, run, and manage data feeds that send logs to your Google Security Operations instance. For details about how to use the Google Security Operations UI to create and manage feeds, see the Feed management user guide.

Prerequisites

Each data feed has its own set of prerequisites that must be completed prior to setting up the feed in Google Security Operations. You can find the prerequisites as follows:

• Prerequisites for each source type are listed in Configuration by source type .

• Prerequisites for each log type ingested using the API feed source type are listed in Configuration by log type .

• Prerequisites for all log types ingested using any source type are listed in the Google Security Operations UI. Go to Settings > Feeds > Add New, select a Source Type and Log Type, and review the required fields. For details, see Creating and editing feeds .

For example, if you set up a data feed from a Google Cloud Storage bucket, you might need to complete the following tasks:

1. Use the feed management fetchFeedServiceAccount method to get a Google Security Operations service account that Google Security Operations uses to ingest data.
2. Grant access to the Google Security Operations service account to the relevant Cloud Storage objects. For more information, see Grant access to the Google Security Operations service account.

How to authenticate with the Google Security Operations API

This Google Security Operations API uses the OAuth 2.0 protocol for authentication and authorization. Your application can complete these tasks using either of the following implementations:

• Using the Google API Client Library for your computer language.

• Directly interfacing with the OAuth 2.0 system using HTTP.

See the reference documentation for the Google Authentication library in Python.

Google Authentication libraries are a subset of the Google API client libraries. See other language implementations.

Getting API authentication credentials

Your Google Security Operations representative will provide you with a Google Developer Service Account Credential to enable the API client to communicate with the API.

You also must provide the Auth Scope when initializing your API client. OAuth 2.0 uses a scope to limit an application's access to an account. When an application requests a scope, the access token issued to the application is limited to the scope granted.

Use the following scope to initialize your Google API client:

https://www.googleapis.com/auth/chronicle-backstory

Python example

The following Python example demonstrates how to use the OAuth2 credentials and HTTP client using google.oauth2 and googleapiclient.

# Imports required for the sample - Google Auth and API Client Library Imports.
# Get these packages from https://pypi.org/project/google-api-python-client/ or run $ pip
# install google-api-python-client from your terminal
from google.auth.transport import requests
from google.oauth2 import service_account

SCOPES = ['https://www.googleapis.com/auth/chronicle-backstory']

# The apikeys-demo.json file contains the customer's OAuth 2 credentials.
# SERVICE_ACCOUNT_FILE is the full path to the apikeys-demo.json file
# ToDo: Replace this with the full path to your OAuth2 credentials
SERVICE_ACCOUNT_FILE = '/customer-keys/apikeys-demo.json'

# Create a credential using Google Developer Service Account Credential and Google Security Operations API
# Scope.
credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)

# Build a requests Session Object to make authorized OAuth requests.
http_session = requests.AuthorizedSession(credentials)

# Your endpoint GET|POST|PATCH|etc. code will vary below

# Reference List example (for US region)
url = 'https://backstory.googleapis.com/v2/lists/COLDRIVER_SHA256'

# You might need another regional endpoint for your API call; see
# https://cloud.google.com/chronicle/docs/reference/ingestion-api#regional_endpoints

# requests GET example
response = http_session.request("GET", url)

# POST example uses json
body = {
  "foo": "bar"
}
response = http_session.request("POST", url, json=body)

# PATCH example uses params and json
params = {
  "foo": "bar"
}
response = http_session.request("PATCH", url, params=params, json=body)

# For more complete examples, see:
# https://github.com/chronicle/api-samples-python/

Chronicle API query limits

The Chronicle API enforces limits on the volume of requests that can be made by any one customer against the Google Security Operations platform. If you reach or exceed the query limit, the Chronicle API server returns HTTP 429 (RESOURCE_EXHAUSTED) to the caller. When developing applications for the Chronicle API, Google Security Operations recommends that you enforce rate limits within your system to avoid resource exhaustion. These limits apply to all of the Chronicle APIs, including the feed management API.

The feed management API enforces the following limits, which are measured in queries per second (QPS):

Control the rate of ingestion

When the data ingestion rate for a tenant reaches a certain threshold, Google Security Operations restricts the rate of ingestion for new data feeds to prevent a source with a high ingestion rate from affecting the ingestion rate of another data source. In this case, there is a delay but no data is lost. The ingestion volume and tenant's usage history determine the threshold.

You can request a rate limit increase by contacting Cloud Customer Care.

Limitations

Data feeds have a maximum log line size of 4 MB.

See the detailed list of the Chronicle API query limits.

Python example using OAuth2 credentials and HTTP client

The following Python example demonstrates how to use the OAuth2 credentials and the HTTP client using google.oauth2 and googleapiclient.

# Imports required for the sample - Google Auth and API Client Library Imports.
# Get these packages from https://pypi.org/project/google-api-python-client/ or
# run $ pip install google-api-python-client from your terminal

from google.auth.transport import requests
from google.oauth2 import service_account

SCOPES = ['https://www.googleapis.com/auth/chronicle-backstory']

# The apikeys-demo.json file contains the customer's OAuth 2 credentials.
# SERVICE_ACCOUNT_FILE is the full path to the apikeys-demo.json file
# ToDo: Replace this with the full path to your OAuth2 credentials

SERVICE_ACCOUNT_FILE = '/customer-keys/apikeys-demo.json'

# Create a credential using Google Developer Service Account Credential and 
Chronicle API Scope.

credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)

# Build an HTTP session to make authorized OAuth requests.

http_session = requests.AuthorizedSession(credentials)

# <your code continues here>

Regional endpoints

Google Security Operations provides regional endpoints for each API.

• Canada—https://northamerica-northeast2-backstory.googleapis.com
• Dammam—https://me-central2-backstory.googleapis.com
• Doha—https://me-central1-backstory.googleapis.com
• Europe Multi-Region—https://europe-backstory.googleapis.com
• Frankfurt—https://europe-west3-backstory.googleapis.com
• London—https://europe-west2-backstory.googleapis.com
• Mumbai—https://asia-south1-backstory.googleapis.com
• Singapore—https://asia-southeast1-backstory.googleapis.com
• Sydney—https://australia-southeast1-backstory.googleapis.com
• Tel Aviv—https://me-west1-backstory.googleapis.com
• Tokyo—https://asia-northeast1-backstory.googleapis.com
• Turin—https://europe-west12-backstory.googleapis.com
• United States Multi-Region—https://backstory.googleapis.com
• Zurich—https://europe-west6-backstory.googleapis.com

Feed Schema API reference

The Feed Schema API returns information that is useful for constructing valid feed management API requests. For example, you can get the data structure representing the entire feed schema. This structure defines the specific fields to specify for each valid combination of feed source type and log type. Alternatively, you can get a list of all log types compatible with a particular feed source type.

Specifically, the feed schema contains:

• Information about each valid feed source type:
  • A human-readable name
  • A human-readable description
  • Whether feeds with a given feed source type can be modified using the API, or are read-only
• Information about each log type:
  • A human-readable name
  • Whether feeds with a given log type can be modified using the API, or are read-only
• Which log types are compatible with which feed source types
• Information about the specific fields to specify for each valid combination of log type and feed source type:
  • A human-readable field name and description
  • Compatibility with other fields
  • Semantic type (e.g. URI, "secret", etc)
  • Whether the field is required
  • What a valid value for the field looks like

The schema can be accessed using a few different methods.

GetFeedSchema

This method returns a structure representing the entire feed schema. The schema is comprised of a list of "feed source type schemas" each of which describe the supported feed source types. Each feed source types' schema contains a list of "log type schemas" that correspond to the set of log types that are compatible with the feed source type and describe the log type. Each log type schema contains a list of "details field schemas" that describe those fields you would set when issuing a Create Feed request, for example, or expect to see as a result of a ListFeed or GetFeed response. The field schemas specified are unique to the particular combination of log type and feed source type.

Request

GET https://backstory.googleapis.com/v1/feedSchema

Response

{
  "feedSourceTypeSchemas": [{
      "name": "feedSourceTypeSchemas/AMAZON_S3",
      "displayName": "Amazon S3",
      "description": "Amazon Simple Storage Service, a service offered by Amazon Web Services that provides object storage through a web service interface",
      "feedSourceType": "AMAZON_S3",
      "logTypeSchemas": [{
          "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL",
          "displayName": "AWS Cloudtrail",
          "logType": "AWS_CLOUDTRAIL",
          "detailsFieldSchemas": [{
              "fieldPath": "details.amazon_s3_settings.authentication.access_key_id",
              "displayName": "Access key ID",
              "description": "An account access key that is a 20-character alphanumeric string, for example AKIAIOSFODNN7EXAMPLE",
              "type": "STRING",
              "exampleInput": "AKIAIOSFODNN7EXAMPLE",
            },
            ...
            {
              "fieldPath": "details.amazon_s3_settings.s3_uri",
              "displayName": "S3 URI",
              "description": "The S3 bucket source URI",
              "type": "STRING_URI",
              "isRequired": true,
              "exampleInput": "s3://cs-prod-cannon-00afe0c847a8/data/",
            }],
        },
        ...
        {
          "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/ABNORMAL_SECURITY",
          "displayName": "Abnormal Security",
          "logType": "ABNORMAL_SECURITY",
          ...
        }],
    },
    ...
    {
      "name": "feedSourceTypeSchemas/AMAZON_SQS",
      "displayName": "Amazon SQS",
      "description": "Amazon Simple Queue Service, a service offered by Amazon Web Services that provides fully managed message queuing service to transfer messages asynchronously",
      "feedSourceType": "AMAZON_SQS",
      ...
    }],
}

ListFeedSourceTypeSchemas

This method returns information about all feed source types.

Request

GET https://backstory.googleapis.com/v1/feedSourceTypeSchemas

Sample Response

{
  "feedSourceTypeSchemas": [{
      "name": "feedSourceTypeSchemas/AMAZON_S3",
      "displayName": "Amazon S3",
      "description": "Amazon Simple Storage Service, a service offered by Amazon Web Services that provides object storage through a web service interface",
      "feedSourceType": "AMAZON_S3",
    },
    ...
    {
      "name": "feedSourceTypeSchemas/AMAZON_SQS",
      "displayName": "Amazon SQS",
      "description": "Amazon Simple Queue Service, a service offered by Amazon Web Services that provides fully managed message queuing service to transfer messages asynchronously",
      "feedSourceType": "AMAZON_SQS",
    }],
}

ListLogTypeSchemas

This method returns information about all log types compatible with a particular feed source type.

Request

GET https://backstory.googleapis.com/v1/feedSourceTypeSchemas/{feed source type}/logTypeSchemas

Sample Request

https://backstory.googleapis.com/v1/feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas

Sample Response

{
  "logTypeSchemas": [{
      "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL",
      "displayName": "AWS Cloudtrail",
      "logType": "AWS_CLOUDTRAIL",
    },
    ...
    {
      "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/ABNORMAL_SECURITY",
      "displayName": "Abnormal Security",
      "logType": "ABNORMAL_SECURITY",
      ...
    }],
}

GetLogTypeSchema

This method returns detailed information about all the fields necessary to configure a feed for a particular source type and log type.

Request

GET https://backstory.googleapis.com/v1/feedSourceTypeSchemas/{feed source type}/logTypeSchemas/{log type}

Sample Request

https://backstory.googleapis.com/v1/feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL

Sample Response

{
  "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL",
  "displayName": "AWS Cloudtrail",
  "logType": "AWS_CLOUDTRAIL",
  "detailsFieldSchemas": [{
      "fieldPath": "details.amazon_s3_settings.authentication.access_key_id",
      "displayName": "Access key ID",
      "description": "An account access key that is a 20-character alphanumeric string, for example AKIAIOSFODNN7EXAMPLE",
      "type": "STRING",
      "exampleInput": "AKIAIOSFODNN7EXAMPLE",
    },
    ...
    {
      "fieldPath": "details.amazon_s3_settings.s3_uri",
      "displayName": "S3 URI",
      "description": "The S3 bucket source URI",
      "type": "STRING_URI",
      "isRequired": true,
      "exampleInput": "s3://cs-prod-cannon-01abc2d345e6/data/",
    }],
}

Feed management API reference

This section describes the endpoints for creating, enabling, and managing feeds.

When creating or editing a feed, you need to specify the feedSourceType and logType in the request body. For details about these fields, see Configuration by source type and Configuration by log type.

Create Feed

Creates a third party data feed in your Google Security Operations instance.

Request

POST https://backstory.googleapis.com/v1/feeds

Request body

This example shows how to collect authentication logs from Duo Security.

{
  "display_name": "some feed name",
  "details": {
    "feedSourceType": "API",
    "logType": "DUO_AUTH",
    "duoAuthSettings": {
      "authentication": {
        "user": "ABCUSERNAMEDEF",
        "secret": "aBcS3cReTdEf"
      },
      "hostname": "api-abc123.duosecurity.com"
    },
    "namespace": "my-asset-namespace",
    "labels": [{
      "key": "my-ingestion-label-key",
      "value": "my-ingestion-label-value"
    }]
  }
}

Sample request

https://backstory.googleapis.com/v1/feeds
{
  "display_name": "some feed name",
  "details": {
    "feedSourceType": "API",
    "logType": "DUO_AUTH",
    "duoAuthSettings": {
      "authentication": {
        "user": "ABCUSERNAMEDEF",
        "secret": "aBcS3cReTdEf"
      },
      "hostname": "api-abc123.duosecurity.com"
    }
  }
}

Sample successful response

{
 "name": "feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "ACTIVE"
}

If the response is unsuccessful, it returns an HTTP status code other than 200 (OK). Be sure to check the body of the response for details of the failure.

Asset namespace

To assign an asset namespace to all events that are ingested from a particular feed, set the "namespace" field within details. The namespace field is a string.

Ingestion label

Ingestion labels are part of Unified Data Model metadata. They are repeated key and value pairs. To assign ingestion labels to all events that are ingested from a particular feed, set the labels field within details. The labels field is an array of JSON objects with key and value fields.

DeleteFeed

Deletes a feed that was configured using the Google Security Operations feed management API.

Request

DELETE  https://backstory.googleapis.com/v1/feeds/{feedID}

Sample request

DELETE https://backstory.googleapis.com/v1/feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567

Sample response

If the operation is successful, Delete Feed returns an empty response with an HTTP status code 200 (OK).

{}

Enable Feed

Enables an INACTIVE feed, which allows it to be executed.

Request

POST https://backstory.googleapis.com/v1/feeds/{feedID}:enable

Sample request

POST https://backstory.googleapis.com/v1/feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567:enable

Sample response

{
 "name": "feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "ACTIVE"
}

Disable Feed

Disables a feed. A disabled feed has a status of INACTIVE. Disabled feeds will no longer fetch data.

Request

POST https://backstory.googleapis.com/v1/feeds/{feedID}:disable

Sample request

POST https://backstory.googleapis.com/v1/feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567:disable

Sample response

{
 "name": "feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "INACTIVE"
}

Get Feed

Gets the details of the feed that was configured.

Request

GET https://backstory.googleapis.com/v1/feeds/{feedID}

Sample request

https://backstory.googleapis.com/v1/feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567

Sample response

{
 "name": "feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "FAILED",
"last_feed_initiation_time": "2024-01-15T01:30:15.01Z",
"failure_details": {
  "error_code": "INVALID_ARGUMENT"
  "http_error_code": 400,
  "error_cause": "A connection to the source was established, but the feed failed because of invalid arguments",
  "error_action":"Check the feed configuration. Learn more about setting up the feeds.\nIf the problem continues, contact Chronicle Support"
 }
}

List Feeds

Retrieves all the feeds configured for a given Google Security Operations instance.

Request

GET https://backstory.googleapis.com/v1/feeds

Sample request

https://backstory.googleapis.com/v1/feeds

Sample response

{
 "feeds": [
   {
     "name": "feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567",
     "details": {
       "logType": "AZURE_AD_CONTEXT",
       "feedSourceType": "API",
       "azureAdContextSettings": {}
     },
     "feedState": "FAILED",
  "last_feed_initiation_time": "2024-01-15T01:30:15.01Z",
  "failure_details": {
    "error_code": "INVALID_ARGUMENT"
    "http_error_code": 400,
    "error_cause": "A connection to the source was established, but the feed failed because of invalid arguments",
    "error_action":"Check the feed configuration. Learn more about setting up the feeds.\nIf the problem continues, contact Chronicle Support"
   }
   },
   {
     "name": "feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567",
     "display_name": "some feed name",
     "details": {
       "logType": "PAN_PRISMA_CLOUD",
       "feedSourceType": "API",
       "panPrismaCloudSettings": {
         "hostname": "api2.prismacloud.io"
       }
     },
     "feedState": "ACTIVE",
  "last_feed_initiation_time": "2024-01-15T01:30:15.01Z",
   }
 ]
}

Read-only feeds

There may be feeds returned from a List Feeds request that have the field readOnly set to true. Read-only feeds cannot be created, updated, or deleted.

Feeds are read-only for a few reasons. For example:

• Some feed source types are not fully supported by feed management at the moment, and were created before the release of feed management.
• Some specialized log types are not available to every Google Security Operations user. If a feed exists with one of these types, it is considered read-only.

Update Feed

Updates the given feed with new details.

Request

PATCH https://backstory.googleapis.com/v1/feeds/{feedID}

Request body

The following examples shows how to update a Duo Auth feed.

Sample request

{
  "display_name": "my feed",
  "details": {
    "feedSourceType": "API",
    "logType": "DUO_AUTH",
    "duoAuthSettings": {
      "authentication": {
        "user": "ABCUSERNAMEDEF",
        "secret": "aBcS3cReTdEf"
      },
      "hostname": "api-abc123.duosecurity.com"
    }
  }
}

Sample response

{
 "display_name": "my feed",
 "name": "feeds/12a34567-bc8d-1111-e9f0-gh1ijk234567",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "ACTIVE"
}

Fetch service account

Gets a unique service account that Google Security Operations uses to ingest data. Use this method only if you're setting up a Cloud Storage feed.

Request

GET https://backstory.googleapis.com/v1/fetchFeedServiceAccount

Sample request

GET https://backstory.googleapis.com/v1/fetchFeedServiceAccount

Sample response

"serviceAccount": "xxxxxxxx-0-account@partnercontent.gserviceyesaccount.com"

Response message fields

This section describes the following fields that are returned in response messages:

• feedState
• failureMsg

Feed state

The feedState field can be found in the response message of most operations. feedState gives some insight into the current state of a feed.

Failure message

The failureMsg field can be found in the response message of most operations, but only for those feeds whose feedState is FAILED. It provides information regarding the error code, cause of the error, and how to troubleshoot the error. For information about error messages, see Troubleshooting. Refer to the following documentation for your particular feed type to understand how to correctly configure the feed.

Generate secret key and API key to authenticate the feed

You need to generate the secret key and API key to authenticate the feed when you set up a feed that has webhook or Amazon Data Firehose as the source type. You can reuse your existing API key to authenticate to Google Security Operations. You must generate a secret key for every new feed and can't reuse the secret key.

1. To generate a secret key for a webhook or Amazon Data Firehose feed, run the following curl command that uses the generateSecret Chronicle API.

      curl --location --request POST -H "Authorization: Bearer $(gcloud auth print-access-token)" 'https://REGIONAL_ENDPOINT/v1alpha/projects/PROJECT_NUMBER/locations/REGION_ID/instances/CUSTOMER_ID/feeds/FEED_ID:generateSecret
   

   Replace the following:

   • REGIONAL_ENDPOINT: the Google Security Operations regional endpoint, such as us-chronicle.googleapis.com. For information about supported regional endpoints, see Google Security Operations regional endpoints section of this document.
   • PROJECT_NUMBER: an automatically generated unique identifier for your project. For information about project name, project ID, and project number, which are used to identify a project, see Creating and managing projects.
   • REGION_ID: the code that Google assigns based on the region. The following are the supported region IDs: Asia-Southeast1, Australia-Southeast1, Europe, EU, Europe-West2, Europe-West3, Europe-West6, Govcloud-US, Me-West1, and US.
   • CUSTOMER_ID: the Google Security Operations customer ID.
   • FEED_ID: the Google Security Operations feed ID.

   A secret key is returned. Copy and store the secret key because you cannot view this secret again. You can use the generateSecret API again to generate a new secret key, but regeneration of the secret key makes the previous secret key obsolete.

2. To generate the API key, do the following:

   1. Go to the Google Cloud console Credentials page.
   2. Click Create credentials, and then select API key.
   3. Restrict the API key access to the Chronicle API.

Configuration by source type

This section provides information about configuring feed source types. A feed source type defines where data is located and how it's accessed. Valid values for feedSourceType are as follows:

API

Use the API feed source type to ingest data from a third-party API. The configuration settings for the API feed source type are specific to the log type that you specify for the feed.

For example, when you use the Feeds API endpoints to configure settings for an API feed data source, you specify API as the value for the details.feedSourceType field. You also specify a supported log type as the value for the details.logType field. The log type determines the additional feed configuration settings that you can specify. Details about these settings are provided in the section for that specific log type in Configuration by log type.

Google Cloud Pub/Sub

Prerequisites

• Ensure that a Google Cloud project for Google Security Operations is configured and the Chronicle API is enabled for the project.
• Link a Google Security Operations instance to Google Cloud services.

Set up push ingestion using Pub/Sub

Data can be sent to Google Security Operations using Pub/Sub. You must first create a feed with the appropriate log type before configuring Pub/Sub to send data.

To set up HTTPS push ingestion using Pub/Sub, do the following:

1. Create a Pub/Sub feed using the following create API request:

     {
       "displayName": "FEED_NAME",
       "details": {
         "feedSourceType": "HTTPS_PUSH_GOOGLE_CLOUD_PUBSUB",
         "logType": "projects/PROJECT_ID/locations/REGION_ID/instances/CUSTOMER_ID/logTypes/LOG_TYPE"
       }
     }
   

   Replace the following:

   • FEED_NAME: specify a name for the feed.
   • PROJECT_ID: the project ID of the project that is bound to Google Security Operations.
   • REGION_ID: the region configured for your Google Security Operations instance. This was set when the tenant was provisioned. The following are the supported region IDs: Asia-Southeast1, Australia-Southeast1, Europe, EU, Europe-West2, Europe-West3, Europe-West6, Govcloud-US, Me-West1, and US.
   • CUSTOMER_ID: the Google Security Operations customer ID.
   • LOG_TYPE: the type of data the feed ingests. Google Security Operations supports specific log types for the Pub/Sub feed.

   If the data to be ingested contains a delimiter that separate log lines, such as \\n, include the following in the details field of the feed request body:

     "httpsPushGoogleCLoudPubsubSettings": {
       "splitDelimiter": "LOG_DELIMITER"
     }
   

   Replace LOG_DELIMITER with the delimiter that separates the log lines, such as \\n.

2. After you create a feed in Pub/Sub, create a push subscription, specify the HTTPS endpoint, and enable authentication. For more information about how to create a push subscription, see Create push subscriptions.

   • Specify the endpoint URL. The endpoint URL must have the following format: https://REGIONAL_ENDPOINT/v1alpha/projects/PROJECT_ID/ locations/REGION_ID/instances/CUSTOMER_ID/feeds/FEED_ID:importPushLogs

     • REGIONAL_ENDPOINT: the Google Security Operations regional endpoint, such as us-chronicle.googleapis.com. For information about supported regional endpoints, see Google Security Operations regional endpoints section of this document.
     • PROJECT_ID: the project ID of the project that is bound to Google Security Operations.
     • REGION_ID: the region configured for your Google Security Operations instance. This was set when the tenant was provisioned. The following are the supported region IDs: Asia-Southeast1, Australia-Southeast1, Europe, EU, Europe-West2, Europe-West3, Europe-West6, Govcloud-US, Me-West1, and US.
     • CUSTOMER_ID: the Google Security Operations customer ID.
     • FEED_ID: the Google Security Operations feed ID.

   • Select Enable authentication and select a service account.

GOOGLE_CLOUD_STORAGE

Prerequisites

Before you set up a Cloud Storage feed, you must get a Google Security Operations service account and provide access to the account so that Google Security Operations can ingest data.

1. Use the feed management fetchFeedServiceAccount method to get a Google Security Operations service account.
2. Grant access to the Google Security Operations service account to the relevant Cloud Storage objects. For more information, see Grant access to the Google Security Operations service account.
3. If VPC Service Controls is enabled, configure an ingress rule to provide access to the Cloud Storage bucket. In this document, see Configure VPC Service Controls.

Recommendation

• If your Cloud Storage bucket contains many small files, it can increase the amount of time it takes to transfer the data. It is recommended to combine smaller files into one large file to accelerate the feed transfer process.

• It is important to set a data retention policy to your Cloud Storage buckets so that the transferred files are deleted from the buckets and are not listed in any future transfer feeds. Alternatively, using the Google SecOps feed management UI, you can set the option to delete the source files from the storage buckets after they have been transferred.

• As Google SecOps pulls files from Cloud Storage on a frequent basis, it is recommended to specify a storage class for the bucket that is the most cost-effective.

Configure VPC Service Controls

If VPC Service Controls is enabled, an ingress rule is required to provide access to the Cloud Storage bucket.

The following Cloud Storage methods must be allowed in the ingress rule:

• google.storage.objects.list. Required for a single file feed.
• google.storage.objects.get. Required for feeds that require directory or subdirectory access.
• google.storage.objects.delete. Required for feeds that require deletion of the source file.

Sample ingress rule

- ingressFrom:
  identities:
    - serviceAccount:8911409095528497-0-account@partnercontent.gserviceaccount.com
  sources:
  - accessLevel: "*"
  ingressTo:
  operations:
  - serviceName: storage.googleapis.com
    methodSelectors:
    - method: google.storage.objects.list
    - method: google.storage.objects.get
    - method: google.storage.objects.delete
  resources:
  - projects/PROJECT_ID

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "GOOGLE_CLOUD_STORAGE",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "gcsSettings": {
     "bucketUri": "gs://bucket/file",
     "sourceType": "FOLDERS_RECURSIVE",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
   }
 }
}

Amazon Data Firehose

Prerequisites

• Ensure that a Google Cloud project for Google Security Operations is configured and the Chronicle API is enabled for the project.
• Link a Google Security Operations instance to Google Cloud services.

Set up push ingestion using Amazon Data Firehose

Data can be sent to Google Security Operations using Amazon Data Firehose. You must first create a feed with the appropriate log type before configuring Amazon Data Firehose to send data.

To set up HTTPS push ingestion using Amazon Data Firehose, do the following:

1. Create an Amazon Data Firehose feed using the following create API request:

     {
       "displayName": "FEED_NAME",
       "details": {
         "feedSourceType": "HTTPS_PUSH_AMAZON_KINESIS_FIREHOSE",
         "logType": "projects/PROJECT_NUMBER/locations/REGION_ID/instances/CUSTOMER_ID/logTypes/LOG_TYPE"
       }
     }
   

   Replace the following:

   • FEED_NAME: specify a name for the feed.
   • PROJECT_NUMBER: an automatically generated unique identifier for your project. For information about project name, project ID, and project number, which are used to identify a project, see Creating and managing projects.
   • REGION_ID: the region configured for your Google Security Operations instance. This was set when the tenant was provisioned. The following are the supported region IDs: Asia-Southeast1, Australia-Southeast1, Europe, EU, Europe-West2, Europe-West3, Europe-West6, Govcloud-US, Me-West1, and US.
   • CUSTOMER_ID: the Google Security Operations customer ID.
   • LOG_TYPE: the type of data the feed ingests. Google Security Operations supports specific log types for the Amazon Data Firehose feed.

   If the data to be ingested contains a delimiter that separate log lines, such as \\n, include the following in the details field of the feed request body:

     "httpsPushAmazonKinesisFirehoseSettings": {
       "splitDelimiter": "LOG_DELIMITER"
     }
   

   Replace LOG_DELIMITER with the delimiter that separates the log lines, such as \\n.

2. After you create a feed, generate a secret key for the feed and generate an API key to authenticate to Google Security Operations.

3. In Amazon Data Firehose, specify the HTTPS endpoint and access key.

   • Specify the endpoint URL. Here is a sample Amazon Data Firehose endpoint: https://REGIONAL_ENDPOINT/v1alpha/projects/PROJECT_NUMBER/ locations/REGION_ID/instances/CUSTOMER_ID/feeds/FEED_ID%3AimportPushLogs?key=API_KEY

     The endpoint includes the following values:

     • REGIONAL_ENDPOINT: the Google Security Operations regional endpoint, such as us-chronicle.googleapis.com. For information about supported regional endpoints, see Google Security Operations regional endpoints section of this document.
     • PROJECT_NUMBER: an automatically generated unique identifier for your project. For information about project name, project ID, and project number, which are used to identify a project, see Creating and managing projects.
     • REGION_ID: the region configured for your Google Security Operations instance. This was set when the tenant was provisioned. The following are the supported region IDs: Asia-Southeast1, Australia-Southeast1, Europe, EU, Europe-West2, Europe-West3, Europe-West6, Govcloud-US, Me-West1, and US.
     • CUSTOMER_ID: the Google Security Operations customer ID.
     • FEED_ID: the Google Security Operations feed ID.
     • API_KEY: the API key value.

   • In the Access key field, specify the secret key that you obtained using the generateSecret API.

AMAZON_S3

Prerequisites

1. Create an S3 bucket.
2. Create a security key for programmatic access.

To learn more about how to configure a feed to ingest data from an Amazon S3 bucket, see Ingest AWS logs.

Recommendation

• If your S3 bucket contains many small files, it can increase the amount of time it takes to transfer the data. It is recommended to combine smaller files into one large file to accelerate the feed transfer process.

• It is important to set a data retention policy to your S3 buckets so that the transferred files are deleted from the buckets and are not listed in any future transfer feeds. Alternatively, using the Google SecOps feed management UI, you can set the option to delete the source files from the storage buckets after they have been transferred.

• As Google SecOps pulls files from S3 buckets on a frequent basis, it is recommended to specify a storage class for your buckets that is the most cost-effective.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "AMAZON_S3",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "amazonS3Settings": {
     "s3Uri": "s3://uri/to/file",
     "sourceType": "FILES",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER",
     "authentication": {
       "region": "US_EAST_1",
       "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
       "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
     },
   }
 }
}

Amazon S3 regions

AMAZON_SQS

You can ingest data from an Amazon SQS service whose entries point to files stored in an Amazon S3 bucket.

When you apply a policy, make sure that you have included sqs:DeleteMessage. Google Security Operations is not able to delete messages if the sqs:DeleteMessage permission is not attached to the SQS queue. All the messages are accumulated on the AWS side, which causes a delay as Google Security Operations repeatedly attempts to transfer the same files.

Amazon SQS regions

Prerequisites

Create the S3 bucket and the SQS queue in the same region.

1. Create an S3 bucket.
2. Create an SQS queue.
   • The queue must be a Standard queue, not a FIFO queue.
3. Set up notifications on your S3 bucket to write to your SQS queue.
   • Be sure to attach an access policy.
4. Create an access key which will be used to access both the SQS queue and the S3 bucket.

To learn more about how to configure a feed to ingest data from an Amazon SQS queue whose entries point to files stored in an Amazon S3 bucket, see Ingest AWS logs.

Recommendation

The use of Amazon SQS is preferred over Amazon S3 as the source type. When an Amazon SQS service is used, Google Security Operations reads the Amazon S3 notifications that are sent to the Amazon SQS service and pulls the corresponding files out of the Amazon S3 bucket. This is effectively a push-based version of an Amazon S3 feed and can be used to reduce the latency of ingestion.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "AMAZON_SQS",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "amazonSqsSettings": {
     "queue": "cs-prod-canon-queue-01234abc56de789f",
     "region": "US_EAST_1",
     "accountNumber": "123456789012",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
     "authentication": {
       "sqsAccessKeySecretAuth": {
         "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
         "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
       },
     },
   }
 }
}

AZURE_BLOBSTORE

Prerequisites

You will need either:

• A shared key authorized to access an Azure Blob Storage container.
• A Shared Access Signature with authority to read an Azure Blob Storage container.

To learn more about how to configure a feed to ingest data from Azure Blob Storage, see Ingest Azure Activity logs.

Recommendation

• If your Azure Blob Storage container contains many small files, it can increase the amount of time it takes to transfer the data. It is recommended to combine smaller files into one large file to accelerate the feed transfer process.

• It is important to set a data retention policy to your Azure Blob Storage containers so that the transferred files are deleted from the containers and are not listed in any future transfer feeds.

• As Google SecOps pulls files from containers on a frequent basis, it is recommended to specify a storage class for your containers that is the most cost-effective.

Type-specific request fields

Azure URI source types

When specifying an Azure URI you must also indicate the type of object is indicated by the URI.

Sample Create Feed request

{
 "details": {
   "feedSourceType": "AZURE_BLOBSTORE",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "azureBlobStoreSettings": {
     "azureUri": "https://myaccount.blob.core.windows.net/logging",
     "sourceType": "FOLDERS_RECURSIVE",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
     "authentication": {
       "sharedKey": "Ab12CyDEFG3HI45JklMnopQrs00TU6xVw7xYZ8AbcdeFgHioJkL0MnoPqRsTUvWxYZaBCdEFg9hijKlm0N12pqR==",
     },
   }
 }
}

HTTP

WARNING: The HTTP type should not be used to gather data from an API. Refer to the following supported API feed types.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "HTTP",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "httpSettings": {
     "uri": "https://url.com/myfile",
     "sourceType": "FILES",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
   }
 }
}

Webhook

Prerequisites

• Ensure that a Google Cloud project for Google Security Operations is configured and the Chronicle API is enabled for the project.
• Link a Google Security Operations instance to Google Cloud services.

Set up push ingestion using an HTTPS webhook

Data can be sent to Google Security Operations using an HTTPS webhook. You must first create a feed with the appropriate log type before configuring an HTTPS webhook to send data.

To set up HTTPS push ingestion using an HTTPS webhook, do the following:

1. Create an HTTPS webhook feed using the following create API request:

     {
       "displayName": "FEED_NAME",
       "details": {
         "feedSourceType": "HTTPS_PUSH_WEBHOOK",
         "logType": "projects/PROJECT_NUMBER/locations/REGION_ID/instances/CUSTOMER_ID/logTypes/LOG_TYPE"
       }
     }
   

   Replace the following:

   • FEED_NAME: specify a name for the feed.
   • PROJECT_NUMBER: an automatically generated unique identifier for your project. For information about project name, project ID, and project number, which are used to identify a project, see Creating and managing projects.
   • REGION_ID: the code that Google assigns based on the region. The following are the supported region IDs: Asia-Southeast1, Australia-Southeast1, Europe, EU, Europe-West2, Europe-West3, Europe-West6, Govcloud-US, Me-West1, and US.
   • CUSTOMER_ID: the Google Security Operations customer ID.
   • LOG_TYPE: the type of data the feed ingests. Google Security Operations supports specific log types for the HTTPS webhook feed.

   If the data to be ingested contains a delimiter that separate log lines, such as \\n, include the following in the details field of the feed request body:

     "httpsPushWebhookSettings": {
       "splitDelimiter": "LOG_DELIMITER"
     }
   

   Replace LOG_DELIMITER with the delimiter that separates the log lines, such as \\n.

2. After you create a feed, generate a secret key for the feed and generate an API key to authenticate to Google Security Operations.

3. In your client application, specify the HTTPS endpoint. Here is a sample HTTPS webhook endpoint:

   https://REGIONAL_ENDPOINT/v1alpha/projects/PROJECT_NUMBER/locations/REGION_ID/instances/CUSTOMER_ID/feeds/FEED_ID:importPushLogs
   

   The endpoint includes the following values:

   • REGIONAL_ENDPOINT: the Google Security Operations regional endpoint, such as us-chronicle.googleapis.com. For information about supported regional endpoints, see Google Security Operations regional endpoints section of this document.
   • PROJECT_NUMBER: an automatically generated unique identifier for your project. For information about project name, project ID, and project number, which are used to identify a project, see Creating and managing projects.
   • REGION_ID: the code that Google assigns based on the region. The following are the supported region IDs: Asia-Southeast1, Australia-Southeast1, Europe, EU, Europe-West2, Europe-West3, Europe-West6, Govcloud-US, Me-West1, and US.
   • CUSTOMER_ID: the Google Security Operations customer ID.
   • FEED_ID: the Google Security Operations feed ID.

4. Enable authentication by specifying the API key and secret key as part of the custom header in the following format: X-goog-api-key = KEY_VALUE X-Webhook-Access-Key = SECRET_VALUE

   Replace the following:

   • SECRET_VALUE: the secret key value that you generated using the GenerateSecret API. You can also pass the secret as a query parameter in the endpoint URL (?secret=SECRET_VALUE). We recommend that you specify the secret as a header instead of specifying it in the URL.
   • KEY_VALUE: the API key value. You can also pass the API key as a query parameter in the endpoint URL (?key=KEY_VALUE). We recommend that you specify the API key as a header instead of specifying it in the URL.

Google Security Operations regional endpoints

When you construct the HTTPS endpoint URL to push feeds, use the following regional endpoints that Google Security Operations supports:

• asia-northeast1-chronicle.googleapis.com
• asia-south1-chronicle.googleapis.com
• asia-southeast1-chronicle.googleapis.com
• australia-southeast1-chronicle.googleapis.com
• eu-chronicle.googleapis.com
• europe-west2-chronicle.googleapis.com
• europe-west3-chronicle.googleapis.com
• europe-west6-chronicle.googleapis.com
• europe-west12-chronicle.googleapis.com
• govcloud-us-chronicle.googleapis.com
• me-central1-chronicle.googleapis.com
• me-central2-chronicle.googleapis.com
• me-west1-chronicle.googleapis.com
• northamerica-northeast2-chronicle.googleapis.com
• us-chronicle.googleapis.com

Configuration by log type

The following table lists the log types that Google Security Operations supports for the API feed source type (that is, ingesting data from third-party APIs).

If a log type has Google Security Operations parser support, the ingested data is stored in Google Security Operations UDM format as well as raw log data.

Click a Data Source name for detailed reference information, prerequisites, and API examples for the log type.

To learn about prerequisites for other log types and feed source types, see Prerequisites.

Anomali ThreatStream

This section provides API reference details for the ANOMALI_IOC log type. For details about the data source, see the Anomali ThreatStream documentation.

Prerequisites

• Get the values for all required request fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "ANOMALI_IOC",
   "anomaliSettings": {
     "authentication": {
       "user": "USERNAME",
       "secret": "APIKEY"
     },
   }
 }
}

AWS EC2 Hosts

This section provides API reference details for the AWS_EC2_HOSTS log type.

Prerequisites

• Get the values for all required request fields.
• Get the following required permissions:
  • The user whose credentials are used to authenticate must have the AmazonEC2ReadOnlyAccess permission.

Type-specific request fields

The following table lists the field values required when creating a feed to collect data for the AWS_EC2_HOSTS log type.

Sample Create Feed request

{
  "details": {
      "awsEc2HostsSettings": {
          "authentication": {
              "user": "AccessKeyID",
              "secret": "SecretAccessKey"
          }
      },
      "feedSourceType": "API",
      "logType": "AWS_EC2_HOSTS"
  }
}

AWS EC2 Instances

This section provides API reference details for the AWS_EC2_INSTANCES log type.

Prerequisites

• Get the values for all required request fields.
• Get the following required permissions:
  • The user whose credentials are used to authenticate must have the AmazonEC2ReadOnlyAccess permission.

Type-specific request fields

The following table lists the field values required when creating a feed to collect data for the AWS_EC2_INSTANCES log type.

Sample Create Feed request

{
  "details": {
      "awsEc2InstancesSettings": {
          "authentication": {
              "user": "AccessKeyID",
              "secret": "SecretAccessKey"
          }
      },
      "feedSourceType": "API",
      "logType": "AWS_EC2_INSTANCES"
  }
}

AWS EC2 VPCs

This section provides API reference details for the AWS_EC2_VPCS log type.

Prerequisites

• Get the values for all required request fields.
• Get the following required permissions:
  • The user whose credentials are used to authenticate must have the AmazonEC2ReadOnlyAccess permission.

Type-specific request fields

The following table lists the field values required when creating a feed to collect data for the AWS_EC2_VPCS log type.

Sample Create Feed request

{
  "details": {
      "awsEc2VpcsSettings": {
          "authentication": {
              "user": "AccessKeyID",
              "secret": "SecretAccessKey"
          }
      },
      "feedSourceType": "API",
      "logType": "AWS_EC2_VPCS"
  }
}

AWS Identity and Access Management

This section provides API reference details for the AWS_IAM log type.

Prerequisites

• Get the values for all required request fields.
• Get the following required permissions:
  • The user whose credentials are used to authenticate must have the IAMReadOnlyAccess permission.

Type-specific request fields

The following table lists the field values required when creating a feed to collect data for the AWS_IAM log type.

Sample Create Feed request

{
  "details": {
      "awsIamSettings": {
          "authentication": {
              "user": "AccessKeyID",
              "secret": "SecretAccessKey"
          },
          "apiType": "USERS"
      },
      "feedSourceType": "API",
      "logType": "AWS_IAM"
  }
}

CrowdStrike Detection Monitoring

This section provides API reference details for the CS_DETECTS log type. For details about the data source, see the CrowdStrike Detection Monitoring documentation.

Prerequisites

• Get the values for all required request fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
  "details": {
    "feedSourceType": "API",
    "logType": "CS_DETECTS",
    "crowdstrikeDetectsSettings": {
      "authentication": {
          "clientId": "CLIENT ID",
          "clientSecret": "CLIENT SECRET",
          "tokenEndpoint": "https://api.us-2.crowdstrike.com/oauth2/token"
      },
      "hostname": "api.crowdstrike.com"
    }
  }
}

Steps to enable Crowdstrike Feed

Create a CrowdStrike API Client

1. Within the CrowdStrike application, create an API client by navigating to Support and resources > API clients and keys.

2. Create a new API Client with API scopes granting permission to Read Detections.

3. Record the values for: Base URL + Client ID + Client Secret. You will need these to set up the Feed in Google Security Operations.

Setup the Google Security Operations feed

1. Launch Google Security Operations, then select the Settings menu.
2. Select Feeds in the left pane and click Add New.
3. Select the Third Party API Source Type and Crowdstrike Detection Monitoring log type.

4. Fill in the requested parameters gathered earlier from CrowdStrike and click Submit.

The feed will begin to retrieve all detections from the CrowdStrike instance in chronological order. Detections older than 6 months will be dropped. After the backfill is complete, which can take some time depending on the number of detections in CrowdStrike, the feed checks for new detections every 5 minutes.

Duo Authentication Logs

This section provides API reference details for the DUO_AUTH log type. For details about the data source, see the Duo Authentication Logs documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Test the endpoint

The Duo Admin API provides programmatic access to the administrative functionality of Duo Security's two-factor authentication platform.

To query your Duo account's authentication logs, you need to send a request to the /admin/v2/logs/authentication endpoint.

For details on how to use the API, see the Authentication Logs section in the Duo documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "DUO_AUTH",
     "duoAuthSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "SECRET"
       },
       "hostname": "api-mytenant.duosecurity.com"
     }
   }
}

Duo Users

This section provides API reference details for the DUO_USER_CONTEXT log type. For details about the data source, see the Duo Users documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "DUO_USER_CONTEXT",
     "duoUserContextSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "SECRET"
       },
       "hostname": "api-mytenant.duosecurity.com"
     }
   }
}

Fidelis Cloud Passage Events

This section provides API reference details for the CLOUD_PASSAGE log type. For details about the data source, see the Cloud Passage Events documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "CLOUD_PASSAGE",
     "cloudPassageSettings": {
       "authentication": {
         "user": "api_key_id",
         "secret": "api_key_secret",
       }
       "eventTypes": [
         "fim_target_integrity_changed",
         "lids_rule_failed",
         "sca_rule_failed"
       ],
     }
   }
}

Fox-IT

This section provides API reference details for the FOX_IT_STIX log type. For details about the data source, see the Fox-IT documentation.

Prerequisites

• Get the values for all authentication and SSL fields.
• Get the following required permissions:
  • None

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "FOX_IT_STIX",
     "foxItStixSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "SECRET"
       },
       "ssl": {
         "sslCertificate": "<cert>",
         "encodedPrivateKey": "key"
       }
       "pollServiceURI": "https://stix.fox-it.com/services/poll",
       "collection": "mycollection"
     }
   }
}

Google Cloud Identity Devices

This section provides API reference details for the GCP_CLOUDIDENTITY_DEVICES log type. For details about the data source, see the Google Cloud Identity Devices documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "GCP_CLOUDIDENTITY_DEVICES",
   "googleCloudIdentityDevicesSettings": {
     "authentication": {
       "tokenEndPoint": "jwt_token_uri",
       "claims": {
         "issuer": "jwt_client_email",
         "subject": "user_email",
         "audience": "jwt_token_uri"
       },
       "rsCredentials": {
         "private_key": "privatekey"
       }
     },
     "apiVersion": "v1",
   }
 }
}

Google Cloud Identity Device Users

This section provides API reference details for the GCP_CLOUDIDENTITY_DEVICEUSERS log type. For details about the data source, see the Google Cloud Identity Device Users documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "GCP_CLOUDIDENTITY_DEVICEUSERS",
   "googleCloudIdentityDeviceUsersSettings": {
     "authentication": {
       "tokenEndPoint": "jwt_token_uri",
       "claims": {
         "issuer": "jwt_client_email",
         "subject": "user_email",
         "audience": "jwt_token_uri"
       },
       "rsCredentials": {
         "private_key": "privatekey"
       }
     },
   }
 }
}

Google Workspace Activities

This section provides API reference details for the WORKSPACE_ACTIVITY log type. For details about the data source, see the Google Workspace Activities documentation.

Prerequisites

In order for Google Security Operations to ingest Google Workspace activities, you must do the following:

1. Enable the Admin SDK API on your Google Cloud project.
2. Create a Service Account which will be used to authenticate against the Admin API.
3. Generate a JSON key for the Service Account.
4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
   • https://www.googleapis.com/auth/admin.reports.audit.readonly
5. Create a Google Workspace user and assign it an admin role which includes the Reports admin privilege, or create a custom role which includes that privilege.
6. Locate your Google Workspace customer ID.

To learn more about how to configure a feed to ingest Google Workspace logs, see Collect Google Workspace logs.

Type-specific request fields

Google Workspace applications

Activities are associated with one or more applications. The applications that Google Security Operations supports includes the following.

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_ACTIVITY",
   "workspaceActivitySettings": {
     "authentication": {
       "tokenEndpoint": "https://oauth2.googleapis.com/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://oauth2.googleapis.com/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
ABCDeFGHIJKLMnopqrsT0u1VWXY...z/abCdefgHIJK+lMN2o345P=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
     "applications": [
       "admin",
       "groups",
       "mobile"
     ],
   }
 }
}

Google Workspace Alerts

This section provides API reference details for the WORKSPACE_ALERTS log type. For details about the data source, see the Google Workspace Alerts documentation.

Prerequisites

For Google Security Operations to ingest Google Workspace alerts, complete the following steps:

1. Enable the Alert Center API on your Google Cloud project.
2. Create a Service Account which will be used to authenticate against the Alert Center API.
3. Generate a JSON key for the Service Account.
4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
   • https://www.googleapis.com/auth/apps.alerts
5. Create a Google Workspace user and assign it an admin role which includes Alert Center view access, or create a custom role which includes that privilege.
6. Locate your Google Workspace customer ID.

To learn more about how to configure a feed to ingest Google Workspace logs, see Collect Google Workspace logs.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_ALERTS",
   "workspaceAlertsSettings": {
     "authentication": {
       "tokenEndpoint": "https://oauth2.googleapis.com/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://oauth2.googleapis.com/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
ABCDeFGHIJKLMnopqrsT0u1VWXY...z/abCdefgHIJK+lMN2o345P=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "1e2x3ample",
   }
 }
}

Google Workspace ChromeOS Devices

This section provides API reference details for the WORKSPACE_CHROMEOS log type. For details about the data source, see the Google Workspace ChromeOS Devices documentation.

Prerequisites

For Google Security Operations to ingest Google Workspace ChromeOS devices, complete the following steps:

1. Enable the Admin SDK API on your Google Cloud project.
2. Create a Service Account which will be used to authenticate against the Admin API.
3. Generate a JSON key for the Service Account.
4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
   • https://www.googleapis.com/auth/admin.directory.device.chromeos.readonly
5. Create a Google Workspace user and assign it an admin role which includes Chrome Management Settings access, or create a custom role which includes that privilege.
6. Locate your Google Workspace customer ID.

To learn more about how to configure a feed to ingest Google Workspace logs, see Collect Google Workspace logs.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_CHROMEOS",
   "workspaceChromeOsSettings": {
     "authentication": {
       "tokenEndpoint": "https://oauth2.googleapis.com/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://oauth2.googleapis.com/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
ABCDeFGHIJKLMnopqrsT0u1VWXY...z/abCdefgHIJK+lMN2o345P=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Google Workspace Groups

This section provides API reference details for the WORKSPACE_GROUPS log type. For details about the data source, see the Google Workspace Groups documentation.

Prerequisites

For Google Security Operations to ingest Google Workspace ChromeOS devices, complete the following steps:

1. Enable the Admin SDK API on your Google Cloud project.
2. Create a Service Account which will be used to authenticate against the Admin API.
3. Generate a JSON key for the Service Account.
4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
   • https://www.googleapis.com/auth/admin.directory.group.readonly
5. Create a Google Workspace user and assign it an admin role which includes Admin API Group read privileges, or create a custom role which includes that privilege.
6. Locate your Google Workspace customer ID.

To learn more about how to configure a feed to ingest Google Workspace logs, see Collect Google Workspace logs.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_GROUPS",
   "workspaceGroupsSettings": {
     "authentication": {
       "tokenEndpoint": "https://oauth2.googleapis.com/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://oauth2.googleapis.com/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
ABCDeFGHIJKLMnopqrsT0u1VWXY...z/abCdefgHIJK+lMN2o345P=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Google Workspace Mobile Devices

This section provides API reference details for the WORKSPACE_MOBILE log type. For details about the data source, see the Google Workspace Mobile Devices documentation.

Prerequisites

In order for Google Security Operations to ingest Google Workspace ChromeOS devices, complete the following steps:

1. Enable the Admin SDK API on your Google Cloud project.
2. Create a Service Account which will be used to authenticate against the Admin API.
3. Generate a JSON key for the Service Account.
4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
   • https://www.googleapis.com/auth/admin.directory.group.readonly
5. Create a Google Workspace user and assign it an admin role which includes Admin API Group read privileges, or create a custom role which includes that privilege.
6. Locate your Google Workspace customer ID.

To learn more about how to configure a feed to ingest Google Workspace logs, see Collect Google Workspace logs.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_MOBILE",
   "workspaceMobileSettings": {
     "authentication": {
       "tokenEndpoint": "https://oauth2.googleapis.com/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://oauth2.googleapis.com/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
ABCDeFGHIJKLMnopqrsT0u1VWXY...z/abCdefgHIJK+lMN2o345P=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Google Workspace Privileges

This section provides API reference details for the WORKSPACE_PRIVILEGES log type. For details about the data source, see the Google Workspace Privileges documentation.

Prerequisites

In order for Google Security Operations to ingest Google Workspace privileges, complete the following steps:

1. Enable the Admin SDK API on your Google Cloud project.
2. Create a Service Account which will be used to authenticate against the Admin API.
3. Generate a JSON key for the Service Account.
4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
   • https://www.googleapis.com/auth/admin.directory.rolemanagement.readonly
5. Create a Google Workspace user and assign it a super admin role.
6. Locate your Google Workspace customer ID.

To learn more about how to configure a feed to ingest Google Workspace logs, see Collect Google Workspace logs.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_PRIVILEGES",
   "workspacePrivilegesSettings": {
     "authentication": {
       "tokenEndpoint": "https://oauth2.googleapis.com/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://oauth2.googleapis.com/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
ABCDeFGHIJKLMnopqrsT0u1VWXY...z/abCdefgHIJK+lMN2o345P=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Google Workspace Users

This section provides API reference details for the WORKSPACE_USERS log type. For details about the data source, see the Google Workspace Users documentation.

Prerequisites

For Google Security Operations to ingest Google Workspace ChromeOS devices, complete the following steps:

1. Enable the Admin SDK API on your Google Cloud project.
2. Create a Service Account which will be used to authenticate against the Admin API.
3. Generate a JSON key for the Service Account.
4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
   • https://www.googleapis.com/auth/admin.directory.user.readonly
5. Create a Google Workspace user and assign it an admin role which includes Admin API User read privileges, or create a custom role which includes that privilege.
6. Locate your Google Workspace customer ID.

To learn more about how to configure a feed to ingest Google Workspace logs, see Collect Google Workspace logs.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_USERS",
   "workspaceUserSettings": {
     "authentication": {
       "tokenEndpoint": "https://oauth2.googleapis.com/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://oauth2.googleapis.com/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
ABCDeFGHIJKLMnopqrsT0u1VWXY...z/abCdefgHIJK+lMN2o345P=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Imperva

This section provides API reference details for the IMPERVA_WAF log type. For details about the data source, see the Imperva documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Optional fields

initialStartTime

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "IMPERVA_WAF",
     "impervaWafSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "key"
            "value": "value"
         }],
       }
     }
   }
}

Microsoft Azure Active Directory Audit

This section provides API reference details for the AZURE_AD_AUDIT log type. For details about the data source, see the Azure Active Directory Audit documentation.

Prerequisites

• Get an Azure AD Premium P1 or P2 license. For more information, see What licenses do I need.
• Get the values for all required request fields. Note that the token endpoint for OAuth 2.0 is: https://login.microsoftonline.com/{tenantId}/oauth2/token
• Get the following required permissions:
  • The user whose credentials are used to authenticate against the Microsoft Graph API to access directory audits must have the permissions AuditLog.Read.All and Directory.Read.All.

Type-specific request fields

Test the API endpoint by using curl

Before you create the feed, you can test the Microsoft Graph API endpoint by using curl.

1. Request an OAuth token to authenticate your request to the API resource.

   curl 'https://login.microsoftonline.com/TENANT_ID/oauth2/token' \
       --data-urlencode 'grant_type=client_credentials' \
       --data-urlencode 'client_id=CLIENT_ID' \
       --data-urlencode 'client_secret=CLIENT_SECRET' \
       --data-urlencode 'resource=https://graph.microsoft.com'
   

   Replace the following:

   • CLIENT_ID: Application ID
   • CLIENT_SECRET: Client secret
   • TENANT_ID: Tenant ID

   The result of the curl request is a JSON response that contains the OAuth access token.

2. Send a request to the Microsoft Graph API endpoint using the OAuth token.

   curl 'https://graph.microsoft.com/v1.0/auditLogs/signIns' \
       --header 'Accept: application/json' \
       --header 'Authorization: Bearer ACCESS_TOKEN'
   

   Replace ACCESS_TOKEN with the value of the OAuth access token that you obtained from the previous step.

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "AZURE_AD_AUDIT",
   "azureAdAuditSettings": {
     "authentication": {
       "clientId": "0ab12c34-d5ef-678g-9012-hi34j56k78l9",
       "clientSecret": "clientSecret",
     }
     "tenantId": "0ab123c4-de56-78fg-90h1-ijk2l3456789",
     "hostname": "graph.microsoft.com/v1.0/auditLogs/directoryAudits",
   }
 }
}

Microsoft Azure Active Directory Organizational Context

This section provides API reference details for the AZURE_AD_CONTEXT log type. For details about the data source, see the Microsoft Graph API List users endpoint, which this feed uses to retrieve device and group data.

Prerequisites

• Get the values for all required request fields. The token endpoint for OAuth 2.0 is https://login.microsoftonline.com/{tenantId}/oauth2/token
• Get the following required permissions:
  • The user whose credentials are used to authenticate against Microsoft Graph API to access organizational context must have permissions Directory.Read.All.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "AZURE_AD_CONTEXT",
   "azureAdContextSettings": {
     "authentication": {
       "clientId": "0ab12c34-d5ef-678g-9012-hi34j56k78l9",
       "clientSecret": "clientSecret",
     }
     "tenantId": "0ab123c4-de56-78fg-90h1-ijk2l3456789",
     "retrieveDevices": false,
     "retrieveGroups": false,
     "hostname": "graph.microsoft.com/beta",
   }
 }
}

Microsoft Azure Active Directory Sign-ins

This section provides API reference details for the AZURE_AD log type. For details about the data source, see the Azure Active Directory Sign-ins documentation.

Prerequisites

• Get an Azure AD Premium P1 or P2 license. For more information, see What licenses do I need.
• Get the values for all required request fields. The token endpoint for OAuth 2.0 is https://login.microsoftonline.com/{tenantId}/oauth2/token
• Get the following required permissions:
  • The user whose credentials are used to authenticate against Microsoft Graph API to access sign-ins must have permissions AuditLog.Read.All and Directory.Read.All.

Type-specific request fields

Sample Create Feed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "AZURE_AD",
   "azureAdSettings": {
     "authentication": {
       "clientId": "0ab12c34-d5ef-678g-9012-hi34j56k78l9",
       "clientSecret": "clientSecret",
     }
     "tenantId": "0ab123c4-de56-78fg-90h1-ijk2l3456789",
     "hostname": "graph.microsoft.com/v1.0/auditLogs/signIns",
   }
 }
}

Microsoft Azure Microsoft Device Management Intune Audit Events

This section provides API reference details for the AZURE_MDM_INTUNE log type. For details about the data source, see the Azure Microsoft Device Management Intune Audit Events documentation.

Prerequisites

• Get an active Intune license.
• Get the values for all authentication fields. The token endpoint for OAuth 2.0 is https://login.microsoftonline.com/{tenantId}/oauth2/token
• Get the following required permission:
  • The provisioned OAuth client must have permission DeviceManagementApps.Read.All or DeviceManagementApps.ReadWrite.All.

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "AZURE_MDM_INTUNE",
     "azureMdmIntuneSettings": {
       "authentication": {
         "clientId": "0ab12c34-d5ef-678g-9012-hi34j56k78l9",
         "clientSecret": "clientSecret",
       }
       "tenantId": "0ab123c4-de56-78fg-90h1-ijk2l3456789",
       "hostname": "graph.microsoft.com/beta/deviceManagement/auditEvents",
     }
   }
}

Microsoft Graph Security API Alerts

This section provides API reference details for the MICROSOFT_GRAPH_ALERT log type. For details about the data source, see Microsoft Graph Security Legacy List alerts and List alerts_v2.

Prerequisites

• Get the values for authentication fields. The token endpoint for OAuth 2.0 is https://login.microsoftonline.com/{tenantId}/oauth2/token
• Get the following required permissions; the API supports two data sources:
  • graph.microsoft.com/v1.0/security/alerts requires SecurityEvents.Read.All permissions
  • graph.microsoft.com/beta/security/alerts_v2 or graph.microsoft.com/v1.0/security/alerts_v2 requires SecurityAlert.Read.All permissions
  • The user whose credentials are used must have permissions SecurityEvents.Read.All.

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "MICROSOFT_GRAPH_ALERT",
     "microsoftGraphAlertSettings": {
       "authentication": {
         "clientId": "0ab12c34-d5ef-678g-9012-hi34j56k78l9",
         "clientSecret": "clientSecret",
       }
       "tenantId": "0ab123c4-de56-78fg-90h1-ijk2l3456789",
       "hostname": "graph.microsoft.com/v1.0/security/alerts",
       "authEndpoint": "login.microsoftonline.com",
     }
   }
}

Microsoft Office 365 Management Activity

This section provides API reference details for the OFFICE_365 log type. For details about the data source, see the Microsoft Office 365 Management Activity documentation.

Prerequisites

• Get the values for all required request fields. The token endpoint for OAuth 2.0 is https://login.microsoftonline.com/{tenantId}/oauth2/token
• Get the following required permissions:
  • The user whose credentials are used to authenticate against the API must have permissions ActivityFeed.Read. If ingesting DLP data then the permission ActivityFeed.ReadDlp must be specified.

To learn more about how to configure a feed to ingest Microsoft Office 365 logs, see Collect Microsoft 365 logs.

Type-specific request fields

Office 365 Content Type

This section provides API reference details for the OFFICE_365 log type. For details about the data source, see the Office 365 Content Type documentation.

Test the API endpoint by using curl

Before you create the feed, you can test the Office 365 Management Activity API endpoint by using curl.

1. Request an OAuth token to authenticate your request to the API resource.

   curl 'https://login.microsoftonline.com/TENANT_ID/oauth2/token' \
       --data-urlencode 'grant_type=client_credentials' \
       --data-urlencode 'client_id=CLIENT_ID' \
       --data-urlencode 'client_secret=CLIENT_SECRET' \
       --data-urlencode 'resource=https://manage.office.com'
   

   Replace the following:

   • CLIENT_ID: Application ID
   • CLIENT_SECRET: Client secret
   • TENANT_ID: Tenant ID

   The result of the curl request is a JSON response that contains the OAuth access token.

2. Send a request to the Office 365 Management Activity API using the OAuth token.

   curl 'https://manage.office.com/api/v1.0/TENANT_ID/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory' \
       --header 'Authorization: Bearer ACCESS_TOKEN'
   

   Replace ACCESS_TOKEN with the value of the OAuth access token that you obtained from the previous step.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "OFFICE_365",
     "office365Settings": {
       "authentication": {
         "clientId": "0ab12c34-d5ef-678g-9012-hi34j56k78l9",
         "clientSecret", "clientSecret",
       },
       "tenantId": "0ab123c4-de56-78fg-90h1-ijk2l3456789"",
       "contentType": "AUDIT_AZURE_ACTIVE_DIRECTORY",
       "hostname": "manage.office.com/api/v1.0",
     }
   }
}

Mimecast

This section provides API reference details for the MIMECAST_MAIL log type. For details about the data source, see the Mimecast documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Test the endpoint

The API endpoint used to download Mimecast MTA logs is /api/audit/get-siem-logs. To use this endpoint, send a POST request to /api/audit/get-siem-logs.

For details on how to use the API, see the sample code in the Mimecast documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "MIMECAST_MAIL",
     "mimecastMailSettings": {
       "authentication": {
         "headerKeyValues": [
           {
             "key": "access_key",
             "value": "ACCESS_KEY"
           },
           {
             "key": "app_id",
             "value": "APP_ID"
           },
           {
             "key": "app_key",
             "value": "APP_KEY"
           },
           {
             "key": "secret_key",
             "value": "SECRET_KEY"
           }
         ]
       },
       "hostname": "xx-api.mimecast.com"
     }
   }
}

Netskope Alerts V1

This section provides API reference details for the NETSKOPE_ALERT log type. For details about the data source, see the Netskope Alerts documentation. Netskope REST API v1 data is supported.

Prerequisites

• Get the values for all authentication fields. Use auth tokens for the Netskope REST API v1.
• Get the following required permissions:
  • None

Type-specific request fields

Optional fields

initialStartTime

Test the API endpoint

Before you create the feed, you can test the Netskope alerts API endpoint by sending a POST request to https://TENANT_URL/api/v1/alerts. This endpoint returns alerts generated by Netskope.

The following is an example request using curl:

curl -X POST 'https://TENANT_URL/api/v1/alerts?' \
    -H 'Content-Type: application/json' \
    -d 'timeperiod=86400' \
    -d 'type=Security%20Assessment' \
    -d 'limit=1' \
    -d 'stimeperiod=2592000' \
    -d 'query=%28compliance_standards.standard%20eq%20%27CSA-CCM-3.0.1%27%29' \
    -d 'token=ACCESS_TOKEN'

Replace the following:

• TENANT_URL: URL of your tenant
• ACCESS_TOKEN: OAuth access token

To learn more about the different query parameters that can be used as a part of the request, see the Get Alerts Data page in the Netskope documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "NETSKOPE_ALERT",
     "netskopeAlertSettings": {
       "authentication": {
         "headerKeyValues": [{
          "key": "token",
          "value": secret
         }]
       },
       "hostname": hostname,
       "feedname": feedname,
       "contentType": contenttype
     }
   },
   "display_name": displayname
}

Netskope Alerts V2

This section provides API reference details for the NETSKOPE_ALERT_V2 log type. For details about the data source, see the Netskope Alerts V2 documentation. Netskope REST API v2 data is supported.

• Ingest schedule = Every 10 mins
• details.feedSourceType = API

Replace API_HOSTNAME with the fully qualified domain name of your Netskope REST API v2 endpoint, such as myinstance.goskope.com.

Prerequisites

• Get the values for all authentication fields. Use auth tokens for the Netskope REST API v2.
• Create a Netskope access token following the steps on the REST API v2 Overview page. Note, when creating the Netskope token make sure to select all the relevant endpoint privileges.

Type-specific request fields

Test the API endpoint

Before you create the feed, you can test the Netskope alerts V2 API endpoint by sending a GET request to https://TENANT_URL. This endpoint returns alerts generated by Netskope.

The following is an example request using curl:

curl -X 'GET' \
    'https://TENANT_URL' \
    -H 'accept: application/json' \
    -H 'Netskope-Api-Token: ACCESS_TOKEN'

Replace the following:

• TENANT_URL: URL of one of the Data sources listed in the Data source table.
• ACCESS_TOKEN: OAuth access token (See Prerequisites for details of creating the token.)

To learn more about the different query parameters that can be used as a part of the request, see the Get Alerts Data page in the Netskope documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "NETSKOPE_ALERT_V2",
     "netskopeAlertV2Settings": {
       "authentication": {
         "headerKeyValues": [{
          "key": "Netskope-Api-Token",
          "value": "token_value"
         }]
       },
       "contentTypes": [
          "uba",
          "securityassessment"
       ],
       "hostname": "myinstance.goskope.com",
       "contentCategory": "alerts"
     }
   }
}

Okta System Log

This section provides API reference details for the OKTA log type. For details about the data source, see the Okta System Log documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Test the API endpoint

Before you create the feed, you can test the Okta System Log API endpoint by sending a GET request to OKTA_URL/api/v1/logs. This endpoint returns system log events that can be ingested into a SIEM platform.

The following is an example request to obtain system log events from a particular point of time in the past:

curl -v -X GET \
    -H "Accept: application/json" \
    -H "Content-Type: application/json" \
    -H "Authorization: SSWS API_TOKEN" \
    "https://OKTA_URL/api/v1/logs?since=DATETIME"

Replace the following:

• API_TOKEN: OAuth access token
• OKTA_URL: fully qualified domain name of your Okta instance, such as example.okta.com
• DATETIME: timestamp in UTC format according to ISO 8601, separating date and time with a T. For example: 2024-01-31T00:00:00Z. The API will fetch the logs recorded after the specified timestamp.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "OKTA",
     "oktaSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization",
            "value": "APITOKEN"
          }]
       },
       "hostname": "hostname"
     }
   }
}

Okta Users

This section provides API reference details for the OKTA_USER_CONTEXT log type. For details about the data source, see the Okta Users documentation.

Prerequisites

• Get the values for hostname and all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

managerIdReferenceField is required when you use a non-Okta ID to reference managers. It should be a JSON field path pointing to the field that contains the manager ID in the result of a call to the "users" Okta API.

{
   "details": {
     "feedSourceType": "API",
     "logType": "OKTA_USER_CONTEXT",
     "oktaSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization",
            "value": "APITOKEN"
          }]
       },
       "hostname": "hostname",
       "managerIdReferenceField": "fooId"
     }
   }
}

Palo Alto Networks AutoFocus

This section provides API reference details for the PAN_IOC log type. For details about the data source, see the Palo Alto Networks AutoFocus documentation.

Prerequisites

• Get the values for feedId, feed, and all authentication fields.
• Get the following required permissions:
  • None

Test the endpoint

To get the results for a custom threat indicator feed, you need to send a request to the custom feed resource of the AutoFocus API. The custom feed endpoint is as follows: /IOCFeed/OUTPUT_FEED_ID/OUTPUT_FEED_NAME.

The following is an example request to retrieve threat intelligence:

curl -X GET \
-H "apiKey:API_KEY" \
https://autofocus.paloaltonetworks.com/api/v1.0/IOCFeed/OUTPUT_FEED_ID/OUTPUT_FEED_NAME?limit=MAX_ENTRIES

Replace the following:

• API_KEY: API key tied to your license

• OUTPUT_FEED_ID: custom threat feed ID number

• OUTPUT_FEED_NAME: name of the custom feed

• MAX_ENTRIES: maximum number of indicator entries displayed in the output

For details on how to use the Palo Alto AutoFoucs API, see the Get Custom Threat Indicator Feed documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PAN_IOC",
     "panIocSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "key"
            "value": "value"
         }],
       }
       "feedId": "ID",
       "feed": "feed"
     }
   }
}

Palo Alto Networks Cortex XDR

This section provides API reference details for the CORTEX_XDR log type. For details about the data source, see the Palo Alto Networks Cortex XDR documentation.

Prerequisites

• Get the values for all authentication fields.
• Make sure the API key is an advanced key, not a standard key.
• Get the following required permissions:
  • None

Type-specific request fields

Optional fields

initialStartTime

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "CORTEX_XDR",
     "cortexXdrSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization"
            "value": "api_key"
         },
         {
            "key": "x-xdr-auth-id"
            "value": "api_key_id"
         }
         ],
       },
       "hostname": "api-abcd.xdr.ab.paloaltonetworks.com",
       "endpoint": "incidents"
     }
   }
}

Palo Alto Networks Prisma Cloud Audit Logs

This section provides API reference details for the PAN_PRISMA_CLOUD log type. For details about the data source, see the Palo Alto Networks Prisma Cloud Audit Logs documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Test the endpoints by using curl

Before you create the feed, you can test the API endpoints by using curl. Send a GET request to https://api.prismacloud.io/audit/redlock

The following example returns audit logs for events that took place on the Prisma Cloud platform:

curl -L 'https://api.prismacloud.io/audit/redlock' \
-H 'Accept: application/json; charset=UTF-8' \
-H 'x-redlock-auth: API_KEY_VALUE'

Replace the following:

• API_KEY_VALUE: The Prisma Cloud authentication value is a JSON Web Token (JWT).

Optional fields

timeType, timeAmount, timeUnit

For details about the data source, see the Palo Alto Networks Prisma Cloud Audit Logs documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PAN_PRISMA_CLOUD",
     "panPrismaCloudSettings": {
       "authentication": {
         "user": "user",
         "password": "password"
       },
       "hostname": "api2.prismacloud.io"
     }
   }
}

Proofpoint on Demand

This section provides API reference details for the PROOFPOINT_ON_DEMAND log type. For details about the data source, see the Proofpoint on Demand documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Other fields

proofpointOnDemandSourceDetails

Optional fields

initialStartTime

Test the endpoint

Before you create a real-time email processing log feed, you can test connectivity between your system and the Proofpoint on Demand (PoD) Log API.

The following is an example request to receive uncompressed data:

curl -i --no-buffer \
-H "Connection: Upgrade" \
-H "Upgrade: websocket" \
-H "Host: logstream.proofpoint.com:443" \
-H "Authorization: Bearer ACCESS_TOKEN " \
-H "Sec-WebSocket-Key: KEY" \
-H "SecWebSocket-Version: 13" \
"https://logstream.proofpoint.com:443/v1/stream?cid=CLUSTER_ID&type=message&sinceTime=DATE_TIME"

Replace the following:

• ACCESS_TOKEN: a token provided by Proofpoint for a customer cluster to authenticate with the service.

• KEY: a base64-encoded key used in the WebSocket opening handshake.

• CLUSTER_ID: the cluster ID assigned by Proofpoint.

• DATE_TIME: start time to begin streaming log data, in ISO 8601 format, which includes timezone information. For example: 2018-08-31T00:00:00-0800. The API fetches the logs recorded after the specified timestamp.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PROOFPOINT_ON_DEMAND",
     "proofpointOnDemandSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "clusterId": "ID"
     }
   }
}

Proofpoint TAP

This section provides API reference details for the PROOFPOINT_MAIL log type. For details about the data source, see the Proofpoint SIEM API documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Test the endpoint

Before you create the feed, you can test the Proofpoint TAP SIEM API endpoint by sending a GET request to /v2/siem/all.

To fetch events for all clicks and messages relating to known threats within the specified time period, use a GET request as follows:

curl \
"https://tap-api-v2.proofpoint.com/v2/siem/all?format=syslog&sinceSeconds=SECONDS" \
--user "PRINCIPAL:SECRET" \
-s

Replace the following:

• SECONDS: an integer representing a time window in seconds from the current API server time. For example, 3600.

• PRINCIPAL: Proofpoint service principal to authenticate to the SIEM API.

• SECRET: Proofpoint API secret to authenticate to the SIEM API.

To learn more about the different query parameters that can be used as a part of the request, see the Proopoint TAP SIEM API documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PROOFPOINT_MAIL",
     "proofpointMailSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       }
     }
   }
}

Qualys VM

This section provides API reference details for the QUALYS_VM log type. For details about the data source, see the Qualys VM documentation (PDF).

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Test the endpoints by using curl

Before you create the feed, you can test the API endpoints by using curl.

• To test the endpoint for the Qualys VM Host List API, use the following curl command:

  curl -H "X-Requested-With: Curl Sample" -u "USERNAME:PASSWORD" \
  "https://qualysapi.qg3.apps.qualys.com/api/2.0/fo/asset/host/?action=list"
  

• To test the endpoint for the Qualys VM Host List Detection API, use the following curl command:

  curl -H "X-Requested-With: Curl Sample" -u "USERNAME:PASSWORD" \
  "https://qualysapi.qg3.apps.qualys.com/api/2.0/fo/asset/host/vm/detection/?action=list"
  

Replace the following:

• USERNAME: username of your Qualys account
• PASSWORD: password of your Qualys account

Sample Create Feed request for Qualys VM Host List API

{
   "details": {
     "feedSourceType": "API",
     "logType": "QUALYS_VM",
     "qualysVmSettings": {
       "authentication": {c
         "user": "USERNAME",
         "secret": "PASSWORD"
       },
       "hostname": "qualysapi.qualys.com/api/2.0/fo/asset/host/?action=list"
     }
   }
}

Sample Create Feed request for Qualys VM Host List Detection API

{
   "details": {
     "feedSourceType": "API",
     "logType": "QUALYS_VM",
     "qualysVmSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "PASSWORD"
       },
       "hostname": "qualysapi.qualys.com/api/2.0/fo/asset/host/vm/detection/?action=list"
     }
   }
}

Qualys Scan

This section provides API reference details for the QUALYS_SCAN log type. For details about the data source, see the Qualys VM documentation (PDF).

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • Ensure API access is enabled for the user.

Scan APIs

The Qualys Scan APIs that Google Security Operations supports include the following.

Test the endpoints by using curl

Before you create the feed, you can test the API endpoints by using curl.

• To test the endpoint for the API type SCAN_SUMMARY_OUTPUT, use the following curl command:

  curl -H "X-Requested-With: Curl Sample" -u "USERNAME:PASSWORD" \
  "https://qualysapi.qg3.apps.qualys.com/api/2.0/fo/scan/vm/summary/?action=list&scan_datetime_since=DATETIME"
  

• To test the endpoint for the API type SCAN_COMPLIANCE_OUTPUT, use the following curl command:

  curl -H "X-Requested-With: Curl Sample" -u "USERNAME:PASSWORD" \
  "https://qualysapi.qg3.apps.qualys.com/api/2.0/fo/scan/compliance/?action=list&launched_after_datetime=DATETIME"
  

• To test the endpoint for the API type SCAN_COMPLIANCE_CONTROL_OUTPUT, use the following curl command:

  curl -H "X-Requested-With: Curl Sample" -u "USERNAME:PASSWORD" \
  "https://qualysapi.qg3.apps.qualys.com/api/2.0/fo/compliance/control/?action=list&updated_after_datetime=DATETIME"
  

Replace the following:

• USERNAME: username of your Qualys account
• PASSWORD: password of your Qualys account
• DATETIME: timestamp in UTC format according to ISO 8601, separating date and time with a T. For example: 2024-01-31T18:00:42Z. The API will fetch the logs recorded after the specified timestamp.

Sample Create Feed request for Qualys Scan API

{
   "details": {
     "feedSourceType": "API",
     "logType": "QUALYS_SCAN",
     "qualysScanSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "PASSWORD"
       },
       "hostname": "qualysapi.qualys.com",
       "api_type": "SCAN_SUMMARY_OUTPUT"
     }
   }
}

Rapid7 InsightVM

This section provides API reference details for the RAPID7_INSIGHT log type. For details about the data source, see the Rapid7 InsightVM documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Optional fields

initialStartTime

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "RAPID7_INSIGHT",
     "rapid7InsightSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "X-Api-Key",
            "value": ApiToken "API_TOKEN"
         }],
       },
       "endpoint": "assets"
       "hostname": "us.api.insight.rapid7.com"
     }
   }
}

Replace API_TOKEN with your API token.

Recorded Future

This section provides API reference details for the RECORDED_FUTURE_IOC log type. For details about the data source, see the Recorded Future documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "RECORDED_FUTURE_IOC",
     "recordedFutureIocSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
     }
   }
}

RH-ISAC

This section provides API reference details for the RH_ISAC_IOC log type. For details about the data source, see the RH-ISAC documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Other fields

tags, queueDelay

Optional fields

initialStartTime

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "RH_ISAC_IOC",
     "rhIsacIocSettings": {
       "authentication": {
         "tokenEndPoint": "endpoint",
         "clientId": "clientId",
         "clientSecret": "clientSecret"
       }
     }
   }
}

Salesforce

This section provides API reference details for the SALESFORCE log type. For details about the data source, see the Salesforce documentation.

Prerequisites

• Salesforce Shield is required.
• Get the values for all authentication fields as described in OAuth 2.0 Username-Password Flow for Special Scenarios.
• Get the following required permissions:
  • None

Type-specific request fields

Optional fields

initialStartTime

Test the endpoint

Before you create the feed, you can test the REST API endpoint by sending a GET request to /services/data/vAPI_VERSION/query. The Query resource is used to retrieve field values from a record.

To query event monitoring records based on fields, such as LogDate and EventType, use a GET request as follows:

curl https://SUBDOMAIN.my.salesforce.com/services/data/vAPI_VERSION/query \
    -X GET \
    -H "Authorization: Bearer AUTH_TOKEN" \
    -G \
    --data-urlencode "q=SELECT Id, EventType, LogFile, LogDate, LogFileLength FROM EventLogFile WHERE LogDate > Yesterday AND EventType = 'API'"

Replace the following:

• SUBDOMAIN: the subdomain name relevant to the Salesforce instance being accessed.
• API_VERSION: version number of the API endpoint. For example, 60.0.
• AUTH_TOKEN: OAuth access token.

Sample Create Feed request using OAuth password grant

{
   "details": {
     "feedSourceType": "API",
     "logType": "SALESFORCE",
     "salesforceSettings": {
       "authentication": {
         "tokenEndpoint": "endpoint",
         "clientId": "clientId",
         "clientSecret": "clientSecret",
         "user": "user",
         "password": "password"
       },
       "hostname": "hostname"
     }
   }
}

Sample Create Feed request using OAuth JWT grant

{
   "details": {
     "feedSourceType": "API",
     "logType": "SALESFORCE",
     "salesforceSettings": {
       "authentication": {
         "tokenEndpoint": "endpoint",
         "issuer": "clientId",
         "subject": "emailID",
         "audience": "audience",
         "privateKey": "RSAKey"
       },
       "hostname": "hostname"
     }
   }
}

SentinelOne Alert

This section provides API reference details for the SENTINELONE_ALERT log type. For details about the data source, see the SentinelOne Alert documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "SENTINELONE_ALERT",
     "sentineloneAlertSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization",
            "value": "ApiToken"
          }]
       },
       "hostname": "hostname",
       "isAlertApiSubscribed": false
     }
   }
}

ServiceNow CMDB

This section provides API reference details for the SERVICENOW_CMDB log type. For details about the data source, see the ServiceNow CMDB documentation.

Prerequisites

• Get the values for all the required fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "SERVICENOW_CMDB",
     "servicenowCmdbSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "hostname": "hostname",
       "feedname": "feedname"
     }
   }
}

Symantec Event Export

This section provides API reference details for the SYMANTEC_EVENT_EXPORT log type. For details about the data source, see the Symantec Event Export documentation.

Prerequisites

• Get the values for all authentication fields.
• Get the following required permissions:
  • None

Type-specific request fields

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "SYMANTEC_EVENT_EXPORT",
     "symantecEventExportSettings ": {
       "authentication": {
         "tokenEndPoint": "REFRESH TOKEN URI",
         "clientId": "CLIENT ID",
         "clientSecret": "CLIENT SECRET",
         "refreshToken": "REFRESH TOKEN",
       }
     }
   }
}

Thinkst Canary

This section provides API reference details for the THINKST_CANARY log type. For details about the data source, see the Thinkst Canary documentation.

Prerequisites

• Get the values for all the required fields.
• Get the following required permissions:
  • None

Type-specific request fields

Test the API endpoint

Before you create the feed, you can test the Canary API endpoint by sending a GET request to DOMAIN/api/v1/incidents/all.

The following is an example request to get all incidents:

curl https://DOMAIN.canary.tools/api/v1/incidents/all \
  -d auth_token=AUTH_TOKEN \
  -d limit=1 \
  -G

Replace the following:

• DOMAIN: unique hash identifying your Canary Console
• AUTH_TOKEN: OAuth access token

To learn more about the different query parameters that can be used as a part of the request, see the Canary API documentation.

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "THINKST_CANARY",
     "thinkstCanarySettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "hostname": "hostname"
     }
   }
}

ThreatConnect

This section provides API reference details for the THREATCONNECT_IOC log type. For details about the data source, see the ThreatConnect documentation.

Prerequisites

• Get the values for all the required fields.
• Get the following required permissions:
  • None

Type-specific request fields

Other fields

queueDelay

Optional fields

initialStartTime

Sample Create Feed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "THREATCONNECT_IOC",
     "threatConnectIocSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "hostname": "hostname",
       "owners": [{
         "owner"
       }]
     }
   }
}

Workday

This section provides API reference details for the WORKDAY log type. For details about the data source, see the Workday Administrator Guide (Integrations > Workday REST API).

Prerequisites

• In the Workday documentation for configuring OAuth 2.0 for your REST API client, follow the steps in Register API Clients.
• Ensure that the Workday administrator provides you the Get and View permissions for the required security domain policies and provides access to the Workday API endpoints.

Type-specific request fields

Sample Create Feed request

The following sample uses a token endpoint, client ID, client secret, and refresh token:

{
   "details": {
     "feedSourceType": "API",
     "logType": "WORKDAY",
     "workdaySettings": {
       "authentication": {
         "tokenEndpoint": "TokenEndpoint",
         "user": "ClientID",
         "clientSecret": "ClientSecret"
         "refreshToken": "RefreshToken"
       },
       "hostname": "hostname",
       "tenantId": "ID"
     }
   }
}

The following sample uses an access token:

{
   "details": {
     "feedSourceType": "API",
     "logType": "WORKDAY",
     "workdaySettings": {
       "authentication": {
         "secret": "AccessToken"
       },
       "hostname": "hostname",
       "tenantId": "ID"
     }
   }
}