Collect Darktrace logs

Supported in:

This document explains how to ingest Darktrace logs to Google Security Operations using a Bindplane agent. This parser first extracts common fields from syslog messages, then uses conditional logic to handle both CEF and JSON formatted Darktrace logs. It maps extracted fields to the Unified Data Model (UDM) schema, enriching the data with security context and standardizing event categorization for downstream analysis.

Before you begin

Ensure that you have the following prerequisites:

  • Google SecOps instance
  • Windows 2016 or later or Linux host with systemd
  • If running behind a proxy, firewall ports are open
  • Privileged access to Darktrace

Get Google SecOps ingestion authentication file

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Collection Agents.
  3. Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.

Get Google SecOps customer ID

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Profile.
  3. Copy and save the Customer ID from the Organization Details section.

Install the Bindplane agent

Windows installation

  1. Open the Command Prompt or PowerShell as an administrator.

  2. Run the following command:

    msiexec /i `https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi` /quiet

Linux installation

  1. Open a terminal with root or sudo privileges.

  2. Run the following command:

    sudo sh -c `$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)` install_unix.sh

Additional installation resources

For additional installation options, consult the installation guide.

Configure the Bindplane agent to ingest Syslog and send to Google SecOps

  1. Access the Configuration File:
    • Locate the config.yaml file. Typically, it's in the /etc/bindplane-agent/ directory on Linux or in the installation directory on Windows.
    • Open the file using a text editor (for example, nano, vi, or Notepad).

  2. Edit the config.yaml file as follows:

    receivers:
    tcplog:
        # Replace the port and IP address as required
        listen_address: `0.0.0.0:10282`

exporters:
    chronicle/chronicle_w_labels:
        compression: gzip
        # Adjust the path to the credentials file you downloaded in Step 1
        creds: '/path/to/ingestion-authentication-file.json'
        # Replace with your actual customer ID from Step 2
        customer_id: <customer_id>
        endpoint: malachiteingestion-pa.googleapis.com
        # Add optional ingestion labels for better organization
        ingestion_labels:
            log_type: DARKTRACE
            raw_log_field: body

service:
    pipelines:
        logs/source0__chronicle_w_labels-0:
            receivers:
                - tcplog
            exporters:
                - chronicle/chronicle_w_labels

  3. Replace the port and IP address as required in your infrastructure.

  4. Replace <customer_id> with the actual customer ID.

  5. Update /path/to/ingestion-authentication-file.json to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.

Restart the Bindlane agent to apply the changes

  • To restart the Bindplane agent in Linux, run the following command:

    sudo systemctl restart bindplane-agent

  • To restart the Bindplane agent in Windows, you can either use the Services console or enter the following command:

    net stop BindPlaneAgent && net start BindPlaneAgent

Configure Syslog on Darktrace

  1. Sign in to the Darktrace web UI.
  2. Go to Admin > System Config.
  3. Click Verify Alert Settings.
  4. Provide the following configuration details:
    • CEF Syslog Alerts: Select True.
    • CEF Syslog Server: Enter the Bindplane IP Address.
    • CEF Syslog Server Port: Enter the Bindplane port number (for example, 10282).
    • CEF Syslog TCP Alert: Select True.
  5. Click Save.

UDM mapping table

Log field UDM mapping Logic
darktraceUrl security_result.url_back_to_product The value is taken from the darktraceUrl field.
darktrace_host observer.hostname The value is taken from the darktrace_host field if it is not an IP address.
darktrace_ip observer.ip The value is taken from the darktrace_ip field.
darktrace_user observer.user.userid The value is taken from the darktrace_user field.
description security_result.summary, metadata.description The value is taken from the description field.
device.customFields.DT-AUTO.macaddress principal.mac The value is taken from the device.customFields.DT-AUTO.macaddress field.
device.did principal.asset.asset_id The value is taken from the device.did field, converted to a string, and prefixed with Device ID:.
device.firstSeen principal.asset.first_seen_time The value is taken from the device.firstSeen field, converted to a string, and parsed as a UNIX timestamp in milliseconds.
device.hostname principal.hostname, principal.asset.hostname The value is taken from the device.hostname field.
device.ip principal.ip, principal.asset.ip The value is taken from the device.ip field if it matches the IP address format.
device.ips.0.subnet additional.fields.subnet The value is taken from the device.ips.0.subnet field and prefixed with subnet.
device.ips.ip principal.ip, principal.asset.ip The value is taken from the device.ips.ip field for each IP address in the list.
device.lastSeen principal.asset.last_discover_time The value is taken from the device.lastSeen field, converted to a string, and parsed as a UNIX timestamp in milliseconds.
device.macaddress principal.mac The value is taken from the device.macaddress field.
device.objecttype principal.asset.type If the value is device, the UDM field is set to WORKSTATION.
device.sid principal.resource.attribute.labels.sid The value is taken from the device.sid field and converted to a string.
device.typelabel principal.resource.attribute.labels.typelabel The value is taken from the device.typelabel field.
device.typename principal.resource.attribute.labels.typename The value is taken from the device.typename field.
dst target.ip, target.asset.ip The value is taken from the dst field.
dpt target.port The value is taken from the dpt field and converted to an integer.
dvc principal.ip, principal.asset.ip If the value of dvc is an IP address, it is added to the UDM field.
dvchost principal.hostname, principal.asset.hostname The value is taken from the dvchost field.
endpoint target.url The value is taken from the endpoint field.
event_time metadata.event_timestamp The value is taken from the event_time field and parsed as an ISO8601 timestamp.
externalId metadata.product_log_id The value is taken from the externalId field.
incidentEventUrl principal.url The value is taken from the incidentEventUrl field.
ip principal.ip, principal.asset.ip The value is taken from the ip field if it matches the IP address format.
issue_msg security_result.summary The value is taken from the issue_msg field.
message security_result.description The value is taken from the message field.
method network.http.method The value is taken from the method field.
model.description metadata.description The value is taken from the model.description field.
model.name metadata.product_event_type The value is taken from the model.name field.
model.now.category security_result.severity If the value is critical, the UDM field is set to CRITICAL. If the value is Informational, the UDM field is set to INFORMATIONAL. If the value is Suspicious, the UDM field is set to HIGH and the category is set to NETWORK_SUSPICIOUS.
model.now.description metadata.description The value is taken from the model.now.description field.
model.now.message security_result.description The value is taken from the model.now.message field.
model.now.name metadata.product_event_type The value is taken from the model.now.name field.
model.now.pid principal.process.pid The value is taken from the model.now.pid field and converted to a string.
model.now.uuid principal.user.userid The value is taken from the model.now.uuid field and the event type is set to USER_UNCATEGORIZED.
model.pid principal.process.pid The value is taken from the model.pid field and converted to a string.
model.then.description principal.resource.attribute.labels.Model Then Description The value is taken from the model.then.description field.
model.then.name principal.resource.attribute.labels.Model Then Name The value is taken from the model.then.name field.
model.then.pid principal.resource.attribute.labels.Model Then Pid The value is taken from the model.then.pid field and converted to a string.
model.then.uuid principal.resource.attribute.labels.Model Then UUID The value is taken from the model.then.uuid field.
model.uuid principal.user.userid The value is taken from the model.uuid field and the event type is set to USER_UNCATEGORIZED.
relatedBreaches.0.modelName security_result.description The value is taken from the relatedBreaches.0.modelName field.
score security_result.priority, security_result.priority_details If the value is between 0.8 and 1, the priority is set to HIGH_PRIORITY. If the value is between 0.5 and 0.79, the priority is set to MEDIUM_PRIORITY. If the value is between 0 and 0.49, the priority is set to LOW_PRIORITY. The priority details are set to Score : followed by the value of score converted to a string.
severity security_result.severity If the value is 2, the UDM field is set to MEDIUM. If the value is greater than 2, the UDM field is set to HIGH, is_alert is set to true, and is_significant is set to true.
shost principal.hostname, principal.asset.hostname The value is taken from the shost field.
smac principal.mac The value is taken from the smac field.
src principal.ip, principal.asset.ip The value is taken from the src field.
status network.http.response_code The value is taken from the status field and converted to a string.
summary metadata.description The value is taken from the summary field.
time The value is taken from the time field, converted to a string, and parsed as a UNIX timestamp in milliseconds.
timestamp The value is taken from the timestamp field and parsed as either an ISO8601 timestamp or a UNIX timestamp in milliseconds.
title security_result.summary The value is taken from the title field.
triggeredComponents.ip intermediary.ip The value is taken from the triggeredComponents.ip field if it matches the IP address format.
triggeredComponents.port intermediary.port The value is taken from the triggeredComponents.port field and converted to an integer.
username principal.user.userid The value is taken from the username field.
metadata.vendor_name Set to DARKTRACE.
metadata.product_name Set to DCIP.
metadata.log_type Set to DARKTRACE.
network.ip_protocol Set to TCP if issue_msg doesn't contain UDP. Otherwise set to UDP.
security_result.action Set to BLOCK if status is 401, otherwise set to ALLOW.
security_result.severity Set to INFORMATIONAL.
network.application_protocol Set to HTTP if method is not empty.
metadata.event_type Set to NETWORK_HTTP if method is not empty. Set to USER_LOGIN if description contains logged into \\\\S+ over ssh. Set to NETWORK_CONNECTION if target_ip is not empty. Otherwise, set to STATUS_UPDATE.
extensions.auth.type Set to MACHINE if description contains logged into \\\\S+ over ssh.
security_result.category Set to DATA_EXFILTRATION if issue_msg contains Exfiltration. Set to NETWORK_MALICIOUS if issue_msg contains Compromise. Otherwise, set to NETWORK_SUSPICIOUS.

Changes

2025-01-16

Enhancement:

  • Added support to parse unparsed logs.

2024-11-21

Enhancement:

  • Corrected spelling of email_direciton_error to email_direction_error.

2024-11-21

Enhancement:

  • Corrected spelling of email_direciton_error to email_direction_error.

2024-10-30

Enhancement:

  • Added a Grok pattern to parse the SYSLOG+JSON logs.

2024-10-25

Enhancement:

  • Mapped direction to network.direction for INBOUND, OUTBOUND, or BROADCAST. Otherwise, mapped it to additional.fields.

2024-10-24

Enhancement:

  • Added a new Grok pattern to parse new JSON logs.

2024-10-08

Enhancement:

  • Changed the event_type from USER_UNCATEGORIZED to EMAIL_UNCATEGORIZED when from field is present.
  • from mapped to network.email.from & principal.user.email_addresses.
  • recipients mapped to network.email.to & target.user.email_addresses.
  • Changed subject mapping from metadata.description to network.email.subject.
  • Changed message_id mapping from additional.fields to network.email.mail_id.
  • Changed uuid mapping from principal.user.userid to metadata.product_log_id.

2024-10-07

Enhancement:

  • Mapped filterType under triggeredFilters to additional.fields.
  • When trigger.value has a non IP value, then mapped trigger.value under triggeredFilters to additional.fields.

2024-09-25

Enhancement:

  • Mapped description to metadata.description.
  • Mapped score field to security_result.priority_details.

2024-09-19

Enhancement:

  • Mapped all fields under triggeredFilters to additional.fields.

2024-09-09

Enhancement:

  • Mapped uuid to principal.user.userid.
  • Mapped from to principal.user.email_addresses.
  • Mapped subject to metadata.description.
  • Mapped anomaly_score, tags, link_hosts, and message_id to additional.fields.
  • Mapped recipients to observer.user.email_addresses.
  • Mapped attachment_sha1s and attachment_sha256s to security_result.detection_fields.

2024-08-29

Enhancement:

  • Mapped hostname field to principal.hostname and principal.asset.hostname.
  • Mapped label field to security_result.attribute.label.
  • Mapped ip_address field to principal.ip and principal.asset.ip.
  • Mapped priority field to security_result.priority_details.
  • Mapped priority_level field to security_result.priority.
  • Mapped alert_name field to security_result.rule_name.
  • Mapped message field to security_result.description.
  • Mapped url field to security_result.url_back_to_product.

2024-08-06

Enhancement:

  • When filterType is Destination IP, then mapped triggeredFilter.trigger.value to target.ip.
  • When principal and target machine data is absent but user data is available then mapped metadata.event_type to USER_UNCATEGORIZED.

2024-04-05

-Bug fix:

  • Changed mapping for model.name and model.now.name from principal.user.user_display_name to metadata.product_event_type.
  • When principal machine data and target machine data are present, then changed mapping for metadata.event_type from GENERIC_EVENT or USER_UNCATEGORIZED to NETWORK_CONNECTION, else mapping it to USER_RESOURCE_ACCESS.

2023-12-20

Bug fix:

  • Fixed the flaky results for the mapping sec_result.about.resource.attribute.labels where key is details.

2023-11-20

Enhancement:

  • Parsed subfields in the message field of the raw log.
  • Mapped uuid to principal.user.userid and set metadata.event_type to USER_UNCATEGORIZED when uuid is present.
  • Mapped direction to network.direction.
  • Mapped from to network.email.from.
  • Mapped subject to network.email.subject.
  • Mapped attachment_sha1s, attachment_sha256s, recipients, link_hosts, tags, actions, anomaly_score, message_id to security_result.detection_fields.
  • Mapped url to security_result.url_back_to_product.
  • Mapped severity to security_result.severity.
  • Mapped hostname to principal.hostname.
  • Added on_error to a JSON block to parse unparsed set of JSON logs.
  • Mapped model.pid to principal.process.pid.
  • Mapped model.uuid to principal.user.userid.
  • Mapped model.name to principal.user.user_display_name.
  • Mapped breachUrl to security_result.url_back_to_product.
  • Mapped device.typelabel, device.sid, device.typename to principal.resource.attribute.labels.
  • Mapped device.ip to principal.ip.
  • Mapped device.ips.0.subnet to additional_fields.
  • Mapped device.did to principal.asset.asset_id.
  • Mapped device.customFields.DT-AUTO.macaddress to principal.mac.
  • Mapped device.firstSeen to principal.asset.first_seen_time.
  • Mapped device.device.lastSeen to principal.asset.last_seen_time.
  • Mapped mitreTechniques to security_result.attack_details.techniques.

2023-09-26

Enhancement:

  • Adjusted the parser to support nested JSON.
  • Fixed the parser to handle special characters in the log.
  • Mapped the fields of new log type.

2023-08-29

Enhancement:

  • Mapped details to sec_result.about.resource.attribute.labels.
  • Mapped principal_port_no to principal.port.
  • Mapped ip_protocol to network.ip_protocol.
  • Mapped location to principal.location.country_or_region.
  • Mapped target_host to target.hostname.
  • Mapped target_ip to target.ip.
  • Mapped source_ip to principal.ip.
  • Mapped source_port to principal.port.
  • Mapped dest_ip to target.ip.
  • Mapped dest_port to target.port.
  • Mapped @host to principal.hostname.
  • Mapped uid to principal.user.userid.
  • Mapped note to principal.application.
  • Mapped @type to sec_result.about.resource.attribute.labels.
  • Mapped opcode to sec_result.about.resource.attribute.labels.
  • Mapped trans_id to sec_result.about.resource.attribute.labels.
  • Mapped query_class to sec_result.about.resource.attribute.labels.

2023-07-14

Enhancement:

  • Mapped dvchost to principal.hostname.
  • Mapped deviceMacAddress to principal.mac.
  • Modified mapping of dvc to map to principal.ip only if it's a valid IP address.

2023-03-24

Enhancement:

  • Mapped model.now.category to security_result.severity.
  • Mapped model.now.message to security_result.description.
  • Mapped model.now.description to metadata.description.
  • Mapped model.now.uuid to principal.user.userid.
  • Mapped model.now.pid to principal.process.pid.
  • Mapped model.now.name to principal.user.user_display_name.
  • Mapped score to security_result.priority.
  • Mapped triggeredComponents.port to intermediary.port.
  • Mapped triggeredComponents.ip to intermediary.ip.
  • Mapped device.ip to principal.ip.
  • Mapped device.macaddress to principal.mac.
  • Mapped device.hostname to principal.hostname.
  • Mapped model.then.logic.data.cid, model.now.logic.data.cid, model.now.tags to additional.fields.
  • Mapped Mappedmodel.then.description,model.then.uuid,model.then.name,model.then.pidtoprincipal.resource.attribute.labels`.
  • Modified metadata.event_type from GENERIC_EVENT to STATUS_UPDATE wherver principal.ip or principal.hostname is present.

2022-10-31

Enhancement:

  • Mapped the field time to metadata.event_timestamp.
  • Mapped the field model.description to metadata.description.
  • Mapped the field model.name to principal.user.user_display_name.
  • Mapped the field model.pid to principal.process.pid.
  • Mapped the field device.did to principal.asset.asset_id.
  • Mapped the field device.objecttype to principal.asset.type.
  • Mapped the field device.ips to principal.ip.
  • Mapped the field device.firstSeen to principal.asset.first_seen_time.
  • Mapped the field device.lastSeen to principal.asset.last_discover_time.
  • Mapped the fields device.sid, device.typename and device.typelabel to principal.resource.attribute.labels.
  • Mapped the field model.tags and model.logic.data to additional.fields.
  • Mapped the field breachUrl to security_result.url_back_to_product.
  • Mapped the field mitreTechniques to security_result.detection_fields.
  • Added conditional checks for details.0.0.contents.2.values.0 mapped to principal.port.
  • Dropped the logs having incorrect json format.

2022-10-13

Enhancement:

  • Added grok to parse new json type logs.
  • Mapped category to security_result.severity.
  • Mapped title to security_result.summary.
  • Mapped details.0.0.contents.1.values.0.hostname to principal.hostname.
  • Mapped details.0.0.contents.1.values.0.ip to principal.ip.
  • Mapped details.0.0.contents.2.values.0 to principal.port.
  • Mapped details.0.0.contents.4.values.0 to principal.location.country_or_region.
  • Mapped details.0.1.contents.0.values.0.hostname to target.hostname.
  • Mapped details.0.1.contents.0.values.0.ip to target.ip.
  • Mapped incidentEventUrl to principal.url.
  • Mapped summary to metadata.description.
  • Mapped model.uuid to principal.user.userid.
  • Mapped relatedBreaches.0.modelName to `security_result.description'.

2022-04-22

Enhancement:

  • Added support for issue code being non-numeric in CEF message.

Need more help? Get answers from Community members and Google SecOps professionals.