Store log entries

This document introduces log buckets, which are the containers that Cloud Logging uses to store your log data. It provides information about location, management of the encryption key, and data retention for log buckets. It also highlights where you can use organization policies or default resource settings to control the location and encryption for new log buckets in folders or organizations.

About log buckets

By default, Cloud Logging encrypts customer content stored at rest. Data stored in log buckets by Logging is encrypted using key-encryption keys, a process known as envelope encryption. Access to your logging data requires access to those key-encryption keys. By default, these are Google-owned and Google-managed encryption keys and they don't require any actions on your part.

Your organization might have regulatory, compliance-related, or advanced encryption requirements that our default encryption at rest doesn't provide. To meet your organization's requirements, instead of using Google-owned and Google-managed encryption keys, you can manage your own keys.

Log buckets are regional resources with a fixed location. Google Cloud manages that infrastructure so that your applications are available redundantly across the zones within that region.

The retention period for the data stored by a log bucket depends on the log bucket. This document contains information about data retention.

To query and view the log data stored in a log bucket, you can use the Logs Explorer or the Log Analytics pages of the Google Cloud console. However, the Log Analytics page requires that you upgrade the log bucket to use analytics and that you query one or more log views. Any log bucket can be upgraded to use analytics.

Support for organizations and folders

To help your organization meet compliance and regulatory needs, Logging supports both organization policies and default resource settings:

System-created log buckets

For each Google Cloud project, billing account, folder, or organization, Cloud Logging creates two log buckets, one named _Required and the other named _Default. Unless default resource settings are configured, for these log buckets, these buckets have Google-owned and Google-managed encryption keys and Cloud Logging selects their location.

You can't delete the system-created log buckets.

_Required log bucket

The _Required log bucket stores log entries that are required for compliance or auditing purposes. For this reason, you can't delete this log bucket and you can't modify which log entries are stored in this log bucket. Log entries in this log bucket are retained for 400 days; you can't change this retention period.

The log entries that are stored in the _Required log bucket for a resource also originate in that resource. That is, the _Required log bucket in a Google Cloud project can only store log entries that originate in that project.

The _Required log bucket stores the following types of log entries:

_Default log bucket

The _Default log bucket stores log entries that aren't automatically stored in the _Required log bucket. Because the _Default log bucket is system created, you can't delete it. However, you can modify which log entries are stored in this log bucket.

Cloud Logging retains the log entries in the _Default bucket for 30 days, unless you configure custom retention for the bucket.

For example, this log bucket stores:

User-defined log buckets

You can create user-defined log buckets in any Google Cloud project. When you create a user-defined log bucket, you select the location and set the data retention period. You have the option to provide a customer-managed encryption key.

You can delete user-defined log buckets. To protect against deleting a log bucket that stores log entries that are within their retention period, you can lock the log bucket against updates.

Control access to a log bucket

Identity and Access Management (IAM) permissions and roles control access to log data. For example, you can do all of the following:

  • Grant read and edit access to a log bucket.
  • Grant edit access to a log bucket based on group membership by using tags.
  • Control access to specific fields in a log entry by configuring field-level access on a log bucket.

  • Grant access to a subset of log entries in a log bucket by creating a log view on that log bucket.

    Every log bucket has a default log view, which typically includes every log entry in the log bucket. For the _Default log bucket, the default log view excludes data access log entries.

To give a user the permissions they need to view and analyze log entries, typically one of the following IAM roles is granted:

  • Logs Viewer (roles/logging.viewer) role: Grants access to all log entries in the _Required bucket, and access to the default log view on the _Default bucket.

  • Private Logs Viewer (roles/logging.privateLogViewer) role: Grants access to all logs in the _Required and _Default buckets, including data access logs.

If you create user-defined log buckets or log views on log buckets, then additional permissions are required. For more information about roles, see Access control with IAM.

List of supported regions

Log buckets are regional resources. The infrastructure that stores, indexes, and searches your log entries is located in a specific geographical location. With the exception of log buckets in the global, eu, or us regions, Google Cloud manages the infrastructure so that your applications are available redundantly across the zones within the region of the log bucket.

The following regions are supported by Cloud Logging:

Global

Region name Region description
global

Logs stored in any data centers in the world. Logs might be moved to different data centers. Unlike other global resources in Google Cloud, global log buckets in Cloud Logging don't provide additional redundancy guarantees compared to a regional log bucket.

Multi-regions: EU and US

Region name Region description
eu

Logs stored in any data centers within the European Union. Logs might be moved to different data centers. No additional redundancy guarantees.
us

Logs stored in any data centers within the United States. Logs might be moved to different data centers. No additional redundancy guarantees.

Africa

Region name Region description
africa-south1 Johannesburg

Americas

Region name Region description
northamerica-northeast1 Montréal
northamerica-northeast2 Toronto
northamerica-south1 Mexico
southamerica-east1 São Paulo
southamerica-west1 Santiago
us-central1 Iowa
us-east1 South Carolina
us-east4 North Virginia
us-east5 Columbus
us-south1 Dallas
us-west1 Oregon
us-west2 Los Angeles
us-west3 Salt Lake City
us-west4 Las Vegas

Asia Pacific

Region name Region description
asia-east1 Taiwan
asia-east2 Hong Kong
asia-northeast1 Tokyo
asia-northeast2 Osaka
asia-northeast3 Seoul
asia-south1 Mumbai
asia-south2 Delhi
asia-southeast1 Singapore
asia-southeast2 Jakarta
australia-southeast1 Sydney
australia-southeast2 Melbourne

Europe

Region name Region description
europe-central2 Warsaw
europe-north1 Finland
europe-north2 Stockholm
europe-southwest1 Madrid
europe-west1 Belgium
europe-west2 London
europe-west3 Frankfurt
europe-west4 Netherlands
europe-west6 Zurich
europe-west8 Milan
europe-west9 Paris
europe-west10 Berlin
europe-west12 Turin

Middle East

Region name Region description
me-central1 Doha
me-central2 Dammam
me-west1 Tel Aviv

What's next