Chronicle Dataplane Ingestion API

This document describes using Chronicle Dataplane Ingestion API methods to forward logs directly to your Google Security Operations instance, eliminating the need for additional hardware or software (for example, forwarders) in your environment.

The Chronicle Dataplane Ingestion API service for Google SecOps improves and expands the functionality of existing ingestion management APIs and partner API services, for example, the Chronicle Ingestion API.

The Chronicle Dataplane Ingestion API is a RESTful API with a JSON payload. You can develop systems to call the Chronicle Dataplane Ingestion API methods directly and forward logs to Google SecOps. Managed Security Service Providers (MSSPs) and your Technology Partners can use these API methods to send logs to Google SecOps.

If you've formatted your log data using Google SecOps UDM, you can forward UDM events to your Google SecOps account using the ImportEvents endpoint. Because UDM events are standardized, Google SecOps can more effectively process and interpret the data, improving its ability to detect security compromises and threats within your enterprise. For information about converting raw logs to UDM, see Format log data as UDM.

You can also forward your data to Google SecOps as unstructured logs using the ImportLogs endpoint. Your unstructured log data is normalized within the Google SecOps infrastructure and made available to you through the Google SecOps console. However, some information may be difficult to extract from the unstructured log data and might only be searchable using the Raw Log Scan.

When the Dataplane Ingestion API processes a batch of logs, it assigns a batch ID for those logs. If the batch ID matches an existing batch ID, the new batch of logs is not forwarded to the Google SecOps instance. Deduplication is based on the batch ID, which is identical if the batches contain the same logs.

To use the Dataplane Ingestion API, you need to authenticate. Grant the Chronicle API Editor IAM (Identity and Access Management) role to a user or service account. This role grants the necessary permissions to call the API.

How to authenticate with the Google SecOps Dataplane Ingestion API

This Google Security Operations API uses the OAuth 2.0 protocol for authentication and authorization. Your application can complete these tasks using either of the following implementations:

• Using the Google API Client Library for your computer language.

• Directly interfacing with the OAuth 2.0 system using HTTP.

For the Google Authentication library in Python, see the google-auth reference.

Google Authentication libraries are a subset of the Google API Client Libraries. For details, see Google API Client Libraries.

Get API authentication credentials

Your Google Security Operations representative provides you with a Google Developer service account credential to enable the API client to communicate with the API, and share its credentials (JSON key) so your API client can authenticate to the API.

You also must provide the Auth Scope when initializing your API client. OAuth 2.0 uses a scope to limit an application's access to an account. When an application requests a scope, the access token issued to the application is limited to the scope granted.

Use the following scope to initialize your Google API client:

https://www.googleapis.com/auth/cloud-platform

Python example

The following Python example demonstrates how to use OAuth 2.0 credentials and the HTTP client, with google.oauth2 and googleapiclient.

# Imports required for the sample - Google Auth and API Client Library.
# Install these packages from https://pypi.org/project/google-api-python-client/
# or run from your terminal: pip install google-api-python-client
import json

from google.auth.transport import requests
from google.oauth2 import service_account

SCOPES = [
    "https://www.googleapis.com/auth/cloud-platform",
]

# Create a credential using an ingestion service account credential file
# and the Google Security Operations API Scope.
credentials = service_account.credentials.from_service_account_file( str(credentials_file_path or DEFAULT_CREDENTIALS_FILE))

# Build a requests Session object to make authorized OAuth 2.0 requests.
http_session = requests.AuthorizedSession(credentials)

# Import Logs example (for US region)
parent = f"projects/{project_id}/locations/us/instances/{project_instance}/"
url = 'https://chronicle.googleapis.com/{parent}/events:import'

body = {"inline_source": {"events": [{"udm": json.loads(json_events)[0]}]}}

response = http_session.request("POST", url, json = body)

# For more complete examples, for details see:
# https://github.com/chronicle/api-samples-python/blob/master/ingestion/v1alpha/create_udm_events.py
# https://github.com/google/secops-wrapper/blob/main/src/secops/chronicle/log_ingest.py

The service account credential provided to you corresponds to one Google SecOps instance.

The requests issued to the Dataplane Ingestion API include the project, location, and instance ID in the resource path. The credential used when issuing requests must have the appropriate IAM permissions for the specified instance.

Regional endpoints

Your API endpoint depends on the region where your customer account is provisioned. Google SecOps provides the following list of Regional service endpoints.

API behavior and limitations

• Limitations:

  • Maximum individual log size: 4 MB
  • Maximum batch size (uncompressed): 4 MB
  • Maximum request size: 6 MB
  • Request timeout: 60 seconds
  • Response times for the API endpoints: The API endpoints have a timeout of 60 seconds.

• Best practices:

  • Request timeout: Set to 60 seconds
  • Number of log lines per batch should be less than 100.
  • Batch size should be less than 1 MB.

Dataplane Ingestion API reference

Use the API methods in this section to ingest data into your Google SecOps instance.

ImportEvents

Use this method to import UDM events into Google SecOps.

Reference Documentation: Method: events.import

Full name: projects.locations.instances.events.import

Request

POST https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/events:import

Path parameters

Request body

The following example shows how to import log data in UDM format using the ImportEvents API method.

{
  "parent": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "inlineSource": {
    "events": [{
      "udm": {
        "metadata": {
          "eventTimestamp": "1970-01-01T03:25:45.000000123Z",
          "eventType": "USER_LOGIN",
          "vendorName": "Acme",
          "productName": "Acme SSO"
        },
        "principal": {
          "ip": ["10.1.2.3"]
        },
        "target": {
          "user": {
            "userid": "mary@altostrat.com",
            "userDisplayName": "Mary Jane"
          },
          "application": "Acme Connect"
        },
        "extensions": {
          "auth": {
            "type": "MACHINE",
            "mechanism": ["NETWORK"]
          }
        }
      }
    }, {
      "udm": {
        "metadata": {
          "eventTimestamp": "1970-01-01T03:25:45.000000123Z",
          "eventType": "NETWORK_HTTP",
          "vendorName": "Acme",
          "productName": "Acme Proxy"
        },
        "principal": {
          "hostname": "host0",
          "ip": ["10.1.2.3"],
          "port": 6000
        },
        "target": {
          "hostname": "www.altostrat.com",
          "ip": ["198.51.100.68"],
          "port": 443,
          "url": "www.altostrat.com/images/logo.png"
        },
        "network": {
          "http": {
            "method": "GET",
            "responseCode": 200
          }
        }
      }
    }]
  }
}

Body parameters

Events.udm object fields

Response

You shouldn't receive a response unless there is an error in the method syntax.

Response error codes

See HTTP response codes for common error codes.

ImportLogs

Import log telemetry. Use this method to send unstructured log entries to Google SecOps in batches. Each batch of log data has a maximum size limit of 4 MB (uncompressed).

Reference Documentation: Method: logs.import

Full name: projects.locations.instances.logTypes.logs.import

Request

POST https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/logs:import

Path parameters

Request body

The following example shows how to format AZURE_AD log data from a source file, using the ImportLogs API method.

{
  "parent": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/AZURE_AD",
  "inlineSource": {
    "logs": [{
      "data": "dGVzdC5sb2c=",
      "logEntryTime": "2025-06-25T06:07:51.000001234Z",
      "collectionTime": "2025-06-25T06:07:51.000000123Z",
      "labels": {
        "google": {
          "value": "dev",
          "rbacEnabled": true
        }
      }
    }],
    "sourceFilename": "source_filename.txt"
  }
}

Body parameters

Log object fields

Response

You shouldn't receive a response unless there is an error in the method syntax.

Response error codes

See HTTP response codes for common error codes.

ImportEntities

Use this method to import entities into Google SecOps.

An entity provides additional context about an entity in a UDM event (asset, user, and so on). For example, a PROCESS_LAUNCH event describes that user abc@example.corp launched process shady.exe. The event does not include information that user abc@example.com is a recently terminated employee who administers a server storing finance data. Information stored in one or more entities can add this additional context.

You are limited to 4MB of data per request.

Reference Documentation: Method: entities.import

Full name: projects.locations.instances.entities.import

Request

POST https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/entities:import

Path parameters

Request body

The following example shows how to import entities data using the ImportEntities API method.

{
  "parent": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "inlineSource": {
    "entities": [{
      "metadata": {
        "collectedTimestamp": "1970-01-01T03:25:45.000000124Z",
        "vendorName": "vendor",
        "productName": "product",
        "entityType": "USER"
      },
      "entity": {
        "user": {
          "userid": "google dev",
          "productObjectId": "dev google"
        }
      }
    }, {
      "metadata": {
        "collectedTimestamp": "1970-01-01T03:25:45.000000124Z",
        "vendorName": "vendor",
        "productName": "product",
        "entityType": "USER"
      },
      "entity": {
        "user": {
          "userid": "google dev",
          "productObjectId": "dev google"
        }
      }
    }],
    "logType": "AZURE_AD_CONTEXT"
  }
}

Body parameters

Entity object fields

Response

If successful, the response body is empty.

Response error codes

See HTTP response codes for common error codes.

ListLogTypes

Use this method to retrieve a list of all the supported log types.

Reference Documentation: Method: logTypes.list

Full name: projects.locations.instances.logTypes.list

Request

GET https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/logTypes

Path parameters

Query parameters

Request body

The request body must be empty.

Response

The following example shows the information returned from the ListLogTypes API method.

{
  "logTypes": [{
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/A10_LOAD_BALANCER",
    "displayName": "A10 Load Balancer"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ABNORMAL_SECURITY",
    "displayName": "Abnormal Security"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ABSOLUTE",
    "displayName": "Absolute Mobile Device Management"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ACALVIO",
    "displayName": "Acalvio"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ACTIVE_IDENTITY_HID",
    "displayName": "Active Identity HID"
  }],
  "nextPageToken": "Qp4b7jgMSY3Byb2plY3RzL21hbGFjaGl0ZS1jYXRmb29kLWJ5b3Atc3RhZ2"
}

Response fields

LogType object fields

Response error codes

See HTTP response codes for common error codes.

Error handling

Here's what to do if an HTTP request fails, depending on the HTTP response code:

HTTP response codes

• 400 Bad Request: The request is invalid.
  Check the error message for details. Don't retry without modification.
• 401 Unauthorized: Authentication failed.
  Check your credentials.
• 403 Forbidden: Permission denied.
  Make sure the caller has the correct IAM permissions.
• 404 Not Found: The resource specified in the request doesn't exist.
  Check the resource name.
• 429 Resource Exhausted: Quota limit reached or rate limit exceeded.
  Retry using exponential backoff, with a minimum delay of 30 seconds. If the request continues to fail, see What to do when reaching the burst limit.
• 500 Internal Server Error: An error occurred on the server.
  Retry using exponential backoff, with a minimum delay of 1 second.
• 503 Service Unavailable: The service is temporarily unavailable.
  Retry using exponential backoff, with a minimum delay of 1 second.

View Dataplane HTTP errors

You can view Dataplane HTTP errors for your BYOP projects using the Metrics Explorer in Cloud Monitoring.

To see the HTTP errors for your BYOP project, follow these steps:

1. Go to the Google Cloud console and select your project.
2. Go to Monitoring > Metrics Explorer.
3. Click Select a Metric.
4. Choose Consumed API as the resource type.
5. Choose API request count (serviceruntime.googleapis.com/api/request_count) as the metric.
6. Add filters to narrow down the results:
   • service: Specify the API service name (e.g., compute.googleapis.com).
   • response_code_class: Filter for 4xx or 5xx.
   • method: Specify the API method.
7. You can group the results to analyze the errors, for example by: method, response_code, or service.

Frequently asked questions

Can customers create an allowlist of specific public IP addresses permitted to connect to the collector API endpoint, or is another form of client authorization supported?

Google SecOps doesn't support allowlisting specific IP addresses. Any connection with a valid API key can connect to the collector API endpoint.

Do you support standard HTTP GZIP compression (for example, Accept-Encoding: gzip header)?

Yes. The collector API endpoint supports standard HTTP GZIP compression using the Accept-Encoding: gzip header.