About analytics views

This document describes analytics views and when you might want to create them. It also describes the differences between analytics views and concepts that you might be familiar with, like log views and saved queries.

To learn how to create, query, and manage your analytics views, see Create and query analytics views.

Overview

An analytics view is a resource that can be queried. This resource contains a SQL query, which queries one or more log views.

A log view on a log bucket controls which log entries in the log bucket that you can see. You can query the log view with the Logs Explorer and the Log Analytics pages. In both cases, the format of the data being queried is determined by the LogEntry data structure.

The SQL query contained by an analytics view lets you transform the log data in one or more log views to a custom format. That is, you can filter which log entries in the log view contribute to the query result, and you define the columns of the result. For example, you might discard columns, move data from a nested JSON field to a column, or add columns.

Analytics views aren't materialized views. An analytics view isn't a precomputed view that periodically caches queryresults. Therefore, querying an analytics view is equivalent to querying the log views that the analytics view queries.

Both an analytics view and a saved-query contain a SQL query. However, an analytics view is a resource that can be queried. You can rerun a saved query, but you can't query the result of a saved query.

Types of analytics views

There are two types of analytics views: user-defined and system-defined:

User-defined analytics views are any analytics views that you create. You can query, edit, and delete user-defined analytics views.
System-defined analytics views are analytics views created by Google Cloud services. You can query system-defined analytics views. However, you can't edit or delete them.

For information about listing the analytics views in your Google Cloud project, see List analytics views.

Location of analytics views

The location of an analytics view is determined by the location of the resources it queries. For example, if an analytics view queries a log view that is in the global location, then the location of the analytics view must also be global. When you use the Google Cloud console to create an analytics view, the location is set automatically.

Benefits of analytics views

There primary benefit of an analytics views is that you can transform your log data and create a consistent schema for other queries. However, you might also find that by creating an analytics view you can reduce the effort you spend in writing queries or improve the structure of your queries. Because analytics views aren't materialized views, their use doesn't necessarily reduce query time.

Example: Data transformation

Assume you are analyzing network performance by using VPC Flow Logs. You need to analyze the overall networking performance and also be able to identify specific networks, IP addresses, or hosts. Unfortunately, the information that you need is contained within nested fields in the json_payload field of a log entry.

To extract this information from the log entries, you write the following query and then save the query as an analytics view named network_details:

SELECT
  JSON_VALUE(resource.labels.subnetwork_name) subnetwork_name,
  JSON_VALUE(json_payload.src_instance.vm_name) vm_name,
  JSON_VALUE(json_payload.connection.src_ip) as src_ip,
  JSON_VALUE(json_payload.connection.src_port) as src_port,
  JSON_VALUE(json_payload.connection.dest_ip) as dest_ip,
  JSON_VALUE(json_payload.connection.dest_port) as dest_port,
  CAST(JSON_VALUE(json_payload.bytes_sent) as INT64) as bytes_sent,
  CAST(JSON_VALUE(json_payload.packets_sent) as INT64) as packets_sent
FROM `VIEW`
WHERE
    log_id = "compute.googleapis.com/vpc_flows"
    AND SEARCH(json_payload.reporter, "SRC")

When you want to analyze the network performance, you can now query your analytics view. For example, if you are only interested in the name of the instance and the amount of data sent, then you might write the following query:

SELECT vm_name, bytes_sent, packets_sent,
FROM `analytics_view.my_project.global.network_details`
ORDER BY bytes_sent DESC
LIMIT 100

Example: Base query for API latency analysis

Assume you are tasked with evaluating and summarizing the latency of requests for one-week intervals. Other teams will use the weekly performance analysis data as a basis for other analyses.

The analytics view that you create, and which can be queried by other teams, reports the minimum, maximum, and average latency of completed requests as recorded by log entries in a specific log view:

SELECT week,MIN(took_ms) as min , MAX(took_ms) AS max, AVG(took_ms) AS avg
FROM (
  SELECT TIMESTAMP_TRUNC(timestamp, WEEK) AS week,
  CAST( JSON_VALUE(json_payload, '$."http.resp.took_ms"') AS INT64) as took_ms
  FROM `VIEW`
  WHERE json_payload IS NOT NULL
    AND SEARCH(labels,"frontend")
    AND JSON_VALUE(json_payload.message) = "request complete"
  ORDER BY took_ms DESC, timestamp ASC
)
GROUP BY week ORDER BY week

Required IAM roles and permissions

Because analytics views query log views, to create and query analytics views, your IAM roles must also let you query log views and use Log Analytics. This section lists the IAM roles required to create analytics views, and those required to query log views and to use Log Analytics:

To get the permissions that you need to create, manage, and use analytics views, ask your administrator to grant you the Observability Analytics User (roles/observability.analyticsUser) IAM role on your project.

This predefined role contains the permissions required to create, manage, and use analytics views. To see the exact permissions that are required, expand the Required permissions section:
Required permissions

The following permissions are required to create, manage, and use analytics views:
- observability.analyticsViews.get
- observability.analyticsViews.list
- observability.analyticsViews.create
- observability.analyticsViews.update
- observability.analyticsViews.delete

To get the permissions that you need to query a log view and to use Log Analytics, ask your administrator to grant you the following IAM roles on your project:
- To query the _Required and _Default log buckets: Logs Viewer (roles/logging.viewer)
- To query all log views in a project: Logs View Accessor (roles/logging.viewAccessor)
You can restrict a principal to a specific log view either by adding an IAM condition to the Logs View Accessor role grant made at the project level, or by adding an IAM binding to the policy file of the log view. For more information, see Control access to a log view.

For information about additional roles that you need to query views on user-defined buckets or to query the _AllLogs view of the _Default log bucket, see Cloud Logging roles.

Limitations

The following limitations apply to analytics views:

An analytics view can't query another analytics view.
An analytics view can query multiple log views. However, the log buckets that host the queried log views must be in one location. For example, suppose you have two log buckets, one in us-east1 and the other in asia-east1. You can't create a analytics view that queries log views on those log buckets.
The parent resource of an analytics view must be a Google Cloud project. You can't create an analytics view in folders or organizations.
Linked datasets aren't supported for analytics views. Therefore, you can query analytics views only by using the Log Analytics page. Also, you must run those queries on the default Cloud Logging service.
There is no API support for creating or managing analytics views.
The following limits apply to analytics views:
- Maximum number of analytics views per Google Cloud project: 100
- Per Google Cloud project, the maximum number of analytics views per region: 50
- Per Google Cloud project, the maximum number of regions that can store analytics views: 10