Composite detections overview

This document introduces composite detections and how they can enhance threat detection workflows by correlating outputs from multiple rules.

Composite detections are generated by rules that use detections from other rules as their input—combined with events, metrics, or entity risk signals. These rules are then combined with events, metrics, or entity risk signals to detect complex, multistage threats that individual rules can miss.

Composite detections help analyze events through defined rule interactions and triggers. This improves accuracy, reduces false positives, and provides a comprehensive view of security threats by correlating data from different sources and attack stages.

The following concepts define the building blocks of composite rules and clarify how they function within detection workflows:

• Composite rules: use detections or alerts (or both) as input. Optionally, enrich them with events, metrics, and a wide range of contextual data from the entity graph, such as prevalence data, threat intelligence, or an entity risk score. These rules must always have a match section and can reference meta fields, match variables, and outcome variables from input rules.

• Detection: output generated when a rule's conditions are met.

• Detection-only rules: composite rules that use only detections or alerts as inputs.

When to use composite detections

Composite detections can be useful for achieving the following goals:

• Correlate outcomes of two or more rules (for example, linking a Malware Downloaded detection with a subsequent C2 Beaconing alert from the same host).

• Enrich alerts with related event data.

• Reduce alert fatigue by only triggering a final alert when a noisy, low-confidence detection occurs multiple times or in combination with other suspicious activity.

• Build an alert for a complex, multi-stage attack where each stage is already identified by its own rule.

Benefits of composite detections

Composite detections have the following benefits:

• Unmask multi-stage attacks: Cyberattacks are often multifaceted and interconnected. Composite detection reveals the broader attack narrative by linking seemingly isolated security events. For example, composite detections can identify the full sequence of an attack, such as an initial breach followed by privilege escalation and data exfiltration.

• Tame alert fatigue: Composite rules consolidate and filter noisy alerts, enabling a more focused response. This approach helps prioritize high-impact incidents and reduce overall alert fatigue.

• Enhance detection accuracy: Combine insights from Unified Data Model (UDM) events, rule detections, entity context, User and Entity Behavior Analytics (UEBA) findings, and data tables to build more accurate detection logic.

• Streamline complex logic: Break down complex detection scenarios into manageable, interconnected, and reusable rules to simplify development and maintenance.

• Use in dashboards: Seamlessly integrate composite detections as data sources for Google SecOps dashboards. You can use them to create visualizations that summarize multi-stage attack patterns, making it easier to understand complex risks.

Common use cases and examples

This section lists some common use cases for composite detections.

Track post-login user activity

A primary use case focused on linking a user's login event with subsequent suspicious activities. While a standard multi-event rule can track a short sequence, a composite detection is better for building a comprehensive risk profile of a user's entire session.

• Goal: Correlate a single event, like a high-risk login, with a wide range of subsequent "weak signal" activities over a longer period, such as a full day.

• Example: Create multiple rules that produce lower level detections. Then, use a composite rule with a long match window (for example, 24 hours) to trigger on an initial suspicious login and correlate it with any of the following detections from the same user:

  • A user clearing their command-line history.

  • The creation of a new local administrator account.

  • A large data upload to a personal Cloud Storage site.

Combine with UEBA metrics

This use case leverages existing UEBA metrics as a starting point for a composite detection to find more complex, long-term behavior.

• Goal: Correlate a spike in a UEBA metric with another anomalous activity.

• Example:

  1. A UEBA rule detects an excessive number of login failures for a user.

  2. Another UEBA rule detects a large number of egress bytes from the same user.

  3. A composite detection links these two separate UEBA findings over a period of days to identify a potential account compromise followed by data theft.

Detect data exfiltration attempts

This involves correlating several distinct user actions that, when combined, might indicate an attempt to exfiltrate data.

• Goal: Build a profile of risky data handling by a single user across multiple devices and actions.

• Correlated actions:

  • Logging in from multiple devices (for example, home versus work machine).

  • Accessing more data sources than usual.

  • Simultaneously downloading, printing, and emailing data.

  • Counting how many classified documents a user is touching within a timeframe.

  • Having tendered a letter of resignation.

How composite detections work

When rules meet predefined conditions, they generate detections. These detections can optionally include outcome variables, which capture specific data or event states.

Composite rules use these detections from other rules as part of their inputs. The evaluation can be based on the information from the original rule's meta section, outcome variables, and match variables.

Based on this evaluation, you can use composite rules to create new detections to be used as an intermediary representation for investigation, and alerting with a subsequent rule. This helps correlate multiple factors from different detections to identify complex threats.

For more information about syntax and examples, see composite rules syntax and examples.

Define your strategy

Before you start creating composite rules, plan your strategy to make sure your new rules are effective, efficient, and solve the right problems.

1. Evaluate your current detection strategy. Review your existing rules to identify those that are too noisy, generate a high number of false positives, or are overly complex and difficult to manage.

2. Determine the specific scenarios where composite rules can provide value. This includes detecting multi-stage attacks, correlating multiple low-confidence alerts into a single high-confidence alert, or enriching detections with additional context from other data sources.

3. Based on your evaluation, create an implementation plan. Decide which noisy rules you need to refine, which complex rules you need to simplify, and which new multi-stage detections you need to prioritize.

This defined plan provides a roadmap for creating targeted and effective composite rules. Consider the following high-level strategies for getting the most value out of composite detections while managing technical constraints.

Select the appropriate method

Before building a composite detection, if you can achieve the required outcome with other alternatives. Analyze whether you can identify a complex pattern with an existing UEBA detection. Over-complicating a detection could increase maintenance overhead and consume rule quota.

• Use a composite detection when: your goal is to correlate the final outcomes of two or more different, pre-existing rules. This connects conceptually separate stages of an attack.

  Example: Correlating a detection from a Malware Downloaded rule with a subsequent detection from a C2 Beaconing Detected rule.

• Use an existing UEBA detection when: you want to find when a user or device breaks its normal pattern of activity.

  Example: Automatically detecting that a user has downloaded 100 GB of data today when they normally only download 1 GB.

Manage rule quotas and risk scores

To manage your organization's resources, understand how different rule types impact your rule quota.

• Curated rules don't count against your custom rule quota.

• Composite rules and custom multi-event rules count against your quota.

You can use a curated detection by setting it to detect-only. This allows the curated rule to perform the initial broad detection without producing alerts. You can then use a composite rule to apply specific logic to these findings, providing more value while strategically managing your quota.

Understand the difference between risk and context

When designing detection logic, distinguish between rules that assess risk and rules that provide context.

Risk is the assessment of how dangerous a set of activities is. A rule designed for risk often aggregates multiple contextual events or detections to make a judgment. For example, while a single failed login provides context, a high number of them indicates the risk of a brute-force attack.

Context refers to the factual details surrounding an event. A rule designed for context enriches one event with details from another. For example, while a rule can detect a successful user login, a contextual rule provides the crucial context that this login came from a new and unusual country.

Example: An initial detection can alert you to a potential risk, such as a DNS call to a malicious domain. A composite rule then correlates that alert with event logs in Google SecOps to find the specific command-line process that initiated the call. This enriches the high-level risk alert with critical, actionable context.

Use long match windows strategically

Composite rules configured with long match windows (for example, 14 days) are executed less frequently. Their high latency can make them unsuitable for time-sensitive alerting. Consider using these long-duration windows for detecting slow, persistent adversarial activity over extended periods.

Use detections for visualization

One strategy for managing noisy rules is to turn their output into a visualization on a dashboard. This approach doesn't consume rule quota and can turn high-volume, low-fidelity data into valuable insights.

By setting a rule to detect only and then plotting its detections in a dashboard widget, you can track trends, identify outliers, and gain a high-level audit view of the activity without being overwhelmed by individual alerts.

Example: Track PII data handling

A rule tracks every time a user handles sensitive PII data. Instead of alerting every time, it's set to detect only. A dashboard widget then shows which users are approaching a daily egress limit (for example, 10,000 bytes). This provides a quick audit view of risky behavior without generating constant alerts.

Example: Monitor specific DLP risks:

A widget aggregates the risk scores from a very specific subset of DLP rules. This allows a specific team (for example, the Data Loss Prevention (DLP) administrators) to monitor only the risks that are relevant, filtering out noise from other security domains.

Build composite detections

The following workflow outlines the typical journey for creating a composite rule. For full syntax and details, see composite rules syntax and examples.

1. Define the threat scenario: Define the specific threat you want to detect.

2. Create or identify the input rules: For each stage of the threat scenario, create or identify an input rule that detects the specific activity.

3. Define the join conditions: Determine the common piece of information that links the detections from your input rules, such as rule labels, variables, or detection fields.

4. Build the composite rule: Write the rule that ingests the detections from the input rules.

   • Define the events section, referencing the input rules by their name, ID, or a shared meta label.

   • Define the match section to specify the join key and the time window for the match.

   • Define the condition section to set the condition that must be met for the final alert to fire.

5. Test and deploy the rule chain: We recommend manually running a retrohunt for each rule in the sequence.

   When you use the Test rule feature on a composite rule, it only runs against pre-existing detections that match the rule's input criteria. It does not automatically execute the underlying rules to generate new inputs for the test, which means you cannot validate an entire rule chain in a single action.

   To run a retrohunt for the rule sequence, do the following:

   1. Manually start a retrohunt from the first rule in the sequence.

   2. Wait for it to complete.

   3. Continue with the next rule.

View composite detection findings

You can view composite detection results in the Detections page. An alert is a composite detection when the Inputs column shows Detection as a source and the Detection type column displays an Alert label with a number next to it (for example, Alert (3)).

Note: If you have both SIEM and SOAR, you can also view the results in the Cases tab.

Optimize composite detections

We recommend the following practices for building composite rules.

Optimize for latency

For minimal latency in detection pipelines, use single-event rules whenever possible, such as for the initial trigger. Composite rules can use their detections to perform more complex correlations with other events, entities, or detections, which helps reduce the overall latency.

Use efficient methods to join detections

We recommend using outcome variables, meta labels, and match variables to join detections. These methods provide more deterministic and reliable results than using event samples. Meta labels are particularly flexible because they let you categorize rules so that a composite rule can target any detection with that label.

For example, if multiple rules share the same meta label tactic: exfiltration, you can have a composite rule that targets any detection where the tactic label has the value exfiltration.

Enhance detections with the function library

You can use the YARA-L function library at strategic points within a composite rule to increase signal and add more complex logic.

Manage rule updates

When you update a rule that is used in one or more composite rules, the system automatically creates a new version of the rule. Composite rules automatically use the new version. We recommend testing the entire updated rule sequence to verify the intended behavior.

Limitations

When designing and implementing composite detections, consider the following limitations:

• Composite rules: Google SecOps supports a maximum depth of 10 for composite rules. Depth is the number of rules from a base rule to the final composite rule.

• Detection-only rules: Have a maximum match window of 14 days. However, the following conditions apply:

  • If the rule uses ingested events, entity graph data, data tables, or reference lists, the match window is limited to 48 hours.

  • Detection-only rules are subject to a daily detection limit of 10,000 detections per rule.

• Outcome variables: Each rule is limited to a maximum of 20 outcome variables. Additionally, each repeated outcome variable is limited to 25 values.

• Event samples: Only 10 event samples are stored per event variable in a rule, such as 10 for $e1 and 10 for $e2.

For more information on detection limits, see Detection limits.

What's next

For information on how to build composite detection rules, see composite detection rules.

Need more help? Get answers from Community members and Google SecOps professionals.