Search Best Practices

Supported in:

This document describes Google's recommended best practices for conducting searches using Search. Searches can require substantial computational resources to complete if they are not constructed carefully. Performance also varies depending on the size and complexity of the data in your Google Security Operations instance.

Construct effective Search queries

Each condition must be in the form of udm-field operator value.

For example: principal.hostname = "win-server"

Always try to narrow your time range to the minimum necessary. Google Security Operations can ingest a tremendous amount of data, so limiting the breadth of that data while conducting a search can substantially improve search performance.

You can use regular expressions when conducting searches:

  • Use AND, OR, and NOT.
  • AND is assumed in the absence of the other operators.
  • Use parentheses to modify the order of precedence. There is a maximum limit of 169 logical operators (OR, AND, and NOT) that can be used in the parentheses.
  • Depending on the field type, field operators can include: = != >= > < <=

Alternatively, you can also use the reference lists.

Use nocase as a search modifier

nocase can be used as a modifier to ignore capitalization.

For example, the following search is invalid:

target.user.userid = "TIM.SMITH" nocase

Regular expressions don't work for enumerated fields

You can't use regular expressions for enumerated fields (fields with a range of predefined values) like metadata.event_type or network.ip_protocol.

For example, the following search is invalid:

metadata.eventtype = /NETWORK*/

However, the following search is valid (and approximates what was attempted above):

(metadata.event_type = "NETWORK_CONNECTION" or metadata.event_type = "NETWORK_DHCP")

Using any and all operators in the Events field

In Search, some fields are labeled as repeated, meaning they contain a list of values or message types. Unlike with Rules, repeated fields in Search are always treated with the any operator by default, with no option to specify all.

When the any operator is used, the predicate is evaluated as true if any value in the repeated field satisfies the condition. For example, if you search for principal.ip != "1.2.3.4" and events in your search include both principal.ip = "1.2.3.4" and principal.ip = "5.6.7.8", Search will generate matches. This expands your search to include results that match any of the operators instead of matching with all of them.

Each element in the repeated field is treated individually. If the repeated field is found in events in the search, the events are evaluated for each element in the field. This can cause unexpected behavior, especially when searching using the != operator.

When using the any operator, the predicate is evaluated as true if any value in the repeated field satisfies the condition.

Timestamps use Unix epoch time

Timestamp fields are matched using Unix epoch time (number of seconds that have passed since Thursday 1 January 1970 00:00:00).

When searching for a specific timestamp, the following (in epoch time) is valid:

metadata.ingested_timestamp.seconds = 1660784400

The following timestamp is invalid:

metadata.ingested_timestamp = "2022-08-18T01:00:00Z"

There are certain fields that are excluded from filters, including the following:

  • metadata.id
  • metadata.product_log_id
  • *.timestamp

Because these fields often contain unique values, they often add unnecessary detail, reducing the effectiveness of the search.

Need more help? Get answers from Community members and Google SecOps professionals.