Composite detection rules

Supported in:

Composite detection in Google SecOps involves connecting multiple YARA-L rules. This document explains how to build a composite rule. For more information, see Overview of composite detections.

Understand the rule structure

Composite detection rules are always multi-event rules and follow the same structure and syntax as a single-event rule.

A composite rule has the following essential components:

  • events section: Defines your inputs; the specific detections or events that the rule analyzes.

  • match section: Specifies how the inputs should be connected over a defined time window.

  • condition section: Contains the final logic that determines whether the joined events meet the criteria to trigger an alert.

Define the inputs in the events section

The first step in building a composite detection rule is to define the rule's inputs in the events section. Inputs for composite rules come from collections, which store the detections generated by other queries. Google SecOps provides the following two methods to access data from collections.

Reference detection content with variables or meta labels

To access data from a detection without referencing the original UDM events, you can use outcome variables, match variables, or meta labels. We recommend this approach because it provides greater flexibility and better compatibility across different rule types.

For example, multiple rules can store a string (such as a URL, filename, or registry key) in a common outcome variable if you're looking for that string across different contexts. To access this string from a composite rule, start with detection and locate the relevant information using elements from the Collection resource.

Example: Suppose a detection rule produces the following information:

  • Outcome variable: dest_domain = "cymbal.com"

  • UDM field: target.hostname = "cymbal.com"

In the composite rule, you can access this data using the following paths:

  • detection.detection.outcomes["dest_domain"] to access the dest_domain outcome variable.

  • detection.collection_elements.references.event.target.hostname to access the target.hostname UDM field.

  • detection.time_window.start_time.seconds to access the detection's start timestamp.

The Collection API and the SecurityResult API provide access to both:

  • Detection metadata and outcome values (detection.detection)
  • Underlying UDM events from referenced rules (collection_elements)

Reference detection content with rule ID or name

You can reference a rule by either its name or ID. We recommend this approach when your detection logic depends on specific rules and you want to reduce the data analyzed to only those rule results. Referencing relevant rules by name or ID improves performance and prevents timeouts by reducing the data analyzed. For example, you can directly query fields like target.url or principal.ip from a known previous detection.

  • Reference a rule by rule ID (recommended): Use the detection.detection.rule_id field to reference a rule by ID. You can find the rule ID in the rule's URL in Google SecOps. User-generated rules have IDs in the format ru_UUID, while curated detections have IDs in the format ur_UUID. For example:

    detection.detection.rule_id = "ru_e0d3f371-6832-4d20-b0ad-1f4e234acb2b"

  • Reference a rule by a rule name: use the detection.detection.rule_name field to reference a rule by name. You can specify the exact rule name or use a regular expression to match it. For example:

    • detection.detection.rule_name = "My Rule Name"
    • detection.detection.rule_name = "/PartOfName/"

Join inputs in match section

To connect related detections, events, or entities in a composite rule, define the match section using variables defined in the events section. These variables can include rule labels, outcome variables, match variables, detection fields, or collection elements.

For information about the syntax, see Match section syntax.

Define condition section

Define the condition section to evaluate the results of the match section. If the condition is true, an alert is generated. For information about the syntax, see Condition section syntax.

Apply advanced techniques to composite rules

This section explains how to apply advanced techniques while building composite rules.

Combine events and detections

Composite rules can combine multiple data sources, including UDM events, entity graph data, and detection fields. The following guidelines apply:

  • Use distinct variables per source: Assign unique event variables to each data source (for example, $e for events, $d for detections), where the data source includes events, entities, and detections.

  • Join sources on shared context: Connect data sources using common values, such as user IDs, IP addresses, or domain names in your rule's conditions.

  • Define a match window: Always include a match clause with a time window no longer than 48 hours.

Example: combining events and detections

rule CheckCuratedDetection_with_EDR_and_EG {
  meta:
    author = "noone@cymbal.com"
  events:
    $d.detection.detection.rule_name = /SCC: Custom Modules: Configurable Bad Domain/
    $d.detection.collection_elements.references.event.network.dns.questions.name = $domain
    $d.detection.collection_elements.references.event.principal.asset.hostname = $hostname

    $e.metadata.log_type = "LIMACHARLIE_EDR"
    $e.metadata.product_event_type = "NETWORK_CONNECTIONS"
    $domain = re.capture($e.principal.process.command_line, "\\s([a-zA-Z0-9.-]+\\.[a-zA-Z0-9.-]+)$")
    $hostname = re.capture($e.principal.hostname, "([^.]*)")

    $prevalence.graph.metadata.entity_type = "DOMAIN_NAME"
    $prevalence.graph.metadata.source_type = "DERIVED_CONTEXT"
    $prevalence.graph.entity.hostname = $domain
    $prevalence.graph.entity.domain.prevalence.day_count = 10
    $prevalence.graph.entity.domain.prevalence.rolling_max <= 5
    $prevalence.graph.entity.domain.prevalence.rolling_max > 0

  match:
    $hostname over 1h

  outcome:
    $risk_score = 80
    $CL_target = array($domain)

  condition:
    $e and $d and $prevalence
}

Create sequential composite detections

Sequential composite detections identify patterns of related events where the sequence of detections is important, such as a brute-force login attempt detection, followed by a successful login. These patterns can combine multiple base detections, raw UDM events, or both.

To create a sequential composite detection, you must enforce that order within your rule. To enforce the expected sequence, use one of the following methods:

  • Sliding windows: Define the sequence of detections using sliding windows in your match conditions.

  • Timestamp comparisons: Compare the timestamps of detections within your rule logic to verify that they happen in the selected order.

Example: sequential composite detections

events:
    $d1.detection.detection.rule_name = "fileEvent_rule"
    $userid = $d1.detection.detection.outcomes["user"]
    $hostname = $d1.detection.detection.outcomes["hostname"]

    $d2.detection.detection.rule_name = "processExecution_rule"
    $userid = $d2.detection.detection.outcomes["user"]
    $hostname = $d2.detection.detection.outcomes["hostname"]

    $d3.detection.detection.rule_name = "networkEvent_rule"
    $userid = $d3.detection.detection.outcomes["user"]
    $hostname = $d3.detection.detection.outcomes["hostname"]

$d3.detection.collection_elements.references.event.metadata.event_timestamp.seconds > $d2.detection.collection_elements.references.event.metadata.event_timestamp.seconds

  match:
    $userid over 24h after $d1

Need more help? Get answers from Community members and Google SecOps professionals.