Search best practices
This document describes Google's recommended best practices for using the Search feature in Google Security Operations. Searches can require substantial computational resources if they're not carefully constructed. Performance also varies depending on the size and complexity of the data in your Google SecOps instance.
Construct effective search queries
Each condition must be in the form of
udm-field operator value
.
For example:
principal.hostname = "win-server"
Narrow the time range for your search
Because Google SecOps can ingest a large amount of data during a search, it's recommended that you minimize the time range to narrow the scope and improve search performance.
Use regular expressions in search query
You can use regular expressions when searching data:
- Use
AND
,OR
, andNOT
. AND
is assumed in the absence of the other operators.- Use parentheses to modify the order of precedence. There is a maximum limit
of 169 logical operators (
OR
,AND
, andNOT
) that can be used in the parentheses. - Depending on the field type, field operators can include:
= != >= > < <=
Alternatively, you can use the reference lists.
Use nocase
as a search modifier
nocase
can be used as a modifier to ignore capitalization.
For example, the following search is invalid:
target.user.userid = "TIM.SMITH" nocase
Don't use regular expressions for enumerated fields
You can't use regular expressions for enumerated fields (fields with a range of
predefined values) like metadata.event_type
or
network.ip_protocol
.
The following example is an invalid search:
metadata.eventtype = /NETWORK*/
Whereas, the following example is a valid search:
(metadata.event_type = "NETWORK_CONNECTION" or
metadata.event_type = "NETWORK_DHCP")
Using any and all operators in the Events field
In Search, some fields are labeled as repeated, which means they contain a list
of values or message types. Repeated fields are always treated with the any
operator by default (there's no option to specify
all
).
When the any
operator is used, the predicate is evaluated as true if any value
in the repeated field satisfies the condition. For example, if you search for
principal.ip != "1.2.3.4"
and events in your search include both
principal.ip = "1.2.3.4"
and principal.ip = "5.6.7.8"
, a match is generated. This expands your search to include results that match any of
the operators instead of matching all of them.
Each element in the repeated field is treated individually. If the repeated
field is found in events in the search, the events are evaluated for each
element in the field. This can cause unexpected behavior, especially when
searching using the !=
operator.
When using the any
operator, the predicate is evaluated as true if any value
in the repeated field satisfies the condition.
Timestamps use Unix epoch time
Timestamp fields are matched using Unix epoch time (number of seconds that have passed since Thursday 1 January 1970 00:00:00).
When searching for a specific timestamp, the following (in epoch time) is valid:
metadata.ingested_timestamp.seconds = 1660784400
The following timestamp is invalid:
metadata.ingested_timestamp = "2022-08-18T01:00:00Z"
There are certain fields that are excluded from filters, including the following:
metadata.id
metadata.product_log_id
*.timestamp
Because these fields often contain unique values, they can add unnecessary detail, which may reduce the effectiveness of the search.
Need more help? Get answers from Community members and Google SecOps professionals.