Search best practices

Supported in:

This document describes Google's recommended best practices for using the Search feature in Google Security Operations. Searches can require substantial computational resources if they're not carefully constructed. Performance also varies depending on the size and complexity of the data in your Google SecOps instance.

Construct effective search queries

Each condition must be in the form of udm-field operator value.

For example: principal.hostname = "win-server"

Because Google SecOps can ingest a large amount of data during a search, it's recommended that you minimize the time range to narrow the scope and improve search performance.

Use regular expressions in search query

You can use regular expressions when searching data:

  • Use AND, OR, and NOT.
  • AND is assumed in the absence of the other operators.
  • Use parentheses to modify the order of precedence. There is a maximum limit of 169 logical operators (OR, AND, and NOT) that can be used in the parentheses.
  • Depending on the field type, field operators can include: = != >= > < <=

Alternatively, you can use the reference lists.

Use nocase as a search modifier

nocase can be used as a modifier to ignore capitalization.

For example, the following search is invalid:

target.user.userid = "TIM.SMITH" nocase

Don't use regular expressions for enumerated fields

You can't use regular expressions for enumerated fields (fields with a range of predefined values) like metadata.event_type or network.ip_protocol.

The following example is an invalid search: metadata.eventtype = /NETWORK*/

Whereas, the following example is a valid search: (metadata.event_type = "NETWORK_CONNECTION" or metadata.event_type = "NETWORK_DHCP")

Using any and all operators in the Events field

In Search, some fields are labeled as repeated, which means they contain a list of values or message types. Repeated fields are always treated with the any operator by default (there's no option to specify all).

When the any operator is used, the predicate is evaluated as true if any value in the repeated field satisfies the condition. For example, if you search for principal.ip != "1.2.3.4" and events in your search include both principal.ip = "1.2.3.4" and principal.ip = "5.6.7.8", a match is generated. This expands your search to include results that match any of the operators instead of matching all of them.

Each element in the repeated field is treated individually. If the repeated field is found in events in the search, the events are evaluated for each element in the field. This can cause unexpected behavior, especially when searching using the != operator.

When using the any operator, the predicate is evaluated as true if any value in the repeated field satisfies the condition.

Timestamps use Unix epoch time

Timestamp fields are matched using Unix epoch time (number of seconds that have passed since Thursday 1 January 1970 00:00:00).

When searching for a specific timestamp, the following (in epoch time) is valid:

metadata.ingested_timestamp.seconds = 1660784400

The following timestamp is invalid:

metadata.ingested_timestamp = "2022-08-18T01:00:00Z"

There are certain fields that are excluded from filters, including the following:

  • metadata.id
  • metadata.product_log_id
  • *.timestamp

Because these fields often contain unique values, they can add unnecessary detail, which may reduce the effectiveness of the search.

Need more help? Get answers from Community members and Google SecOps professionals.