Google SecOps data ingestion

Supported in:

Google Security Operations ingests customer logs, normalizes the data, and detects security alerts. It provides self-service features for data ingestion, threat detection, alerts, and case management. Google SecOps can also receive alerts from other SIEM systems and analyze them.

Google SecOps log ingestion

The Google SecOps ingestion service acts as a gateway for all data.

Google SecOps ingests data using the following systems:

  • Forwarders: Remote agents installed on customer endpoints that send data to the Google SecOps ingestion service. For details about how to install Linux and Windows forwarders, see Install and configure the forwarder.

  • Bindplane agent: The Bindplane agent collects logs from various sources and sends them to Google SecOps. You can manage this agent using the optional Bindplane OP Management console. For more information, see Use the Bindplane agent.

  • Ingestion APIs: Google SecOps provides public ingestion APIs, that let you send data directly. For more information, see the Ingestion API.

  • Google Cloud: Google SecOps retrieves data directly from your Google Cloud organization. For more information, see Ingest Google Cloud data to Google SecOps.

  • Data feeds: Data feeds retrieve data from static external locations (such as Amazon S3) and third-party APIs (such as Okta). These data feeds send logs directly to the Google SecOps ingestion service. For more information, see the feed management documentation.

    Data feeds support log lines up to 4 MB in size.

Parsers convert logs from customer systems into a Unified Data Model (UDM). Downstream systems within Google SecOps use the UDM to provide additional capabilities, including rules and UDM search. Google SecOps can ingest both logs and alerts, but supports only single-event alerts. You can use UDM search to find both ingested and built-in Google SecOps alerts.

Understand Google SecOps ingestion process

Google SecOps supports the following types of data ingestion:

Raw logs

Google SecOps ingests raw logs using forwarders, the ingestion API, data feeds, or directly from Google Cloud.

Alerts from other SIEM systems

Google SecOps can ingest alerts from other SIEM systems, EDRs, or ticketing systems, as follows:

  1. Receive alerts using Google SecOps connectors or Google SecOps webhooks.
  2. Ingest the events associated with each alert and create a corresponding detection.
  3. Process both the ingested events and detections.

You can create detection engine rules to identify patterns in the ingested events and generate additional detections.

Need more help? Get answers from Community members and Google SecOps professionals.