Regional resources

When you attach a Cloud Storage bucket as a Dataplex asset, Dataplex creates a publishing dataset in the corresponding region to publish tables discovered in the bucket.

This page describes how Dataplex maps single, dual, and multiple regions in Cloud Storage to BigQuery publishing datasets.

Mapping of Cloud Storage regions to BigQuery datasets

Dataplex lakes, zones, and assets are regional resources, and can reside in one or more regions. BigQuery datasets and Cloud Storage buckets are also regional resources that can reside in one or more regions.

The following are the differences between the regional resources available in Cloud Storage and in BigQuery:

  • Both Cloud Storage and BigQuery support single-region resources.

  • Cloud Storage has dual-regions, whereas BigQuery doesn't.

  • Both Cloud Storage and BigQuery have multi-regions, but they are different.

You can attach Cloud Storage buckets and BigQuery datasets to Dataplex zones or lakes as Dataplex assets. Dataplex automates the creation of publishing datasets for Cloud Storage buckets attached as assets.

Dataplex ensures that the BigQuery and Cloud Storage regions match. If there is no overlap between your Dataplex lake's region and one of the Cloud Storage bucket's regions, you can't add the bucket to your lake's zone.

  • In the case of a single-region Cloud Storage bucket, Dataplex creates a single-region publishing dataset in the same region as the bucket.

  • In the case of the Cloud Storage bucket located in either the Cloud Storage us multi-region or the Cloud Storage eu multi-region, Dataplex creates a publishing dataset in the corresponding BigQuery us or eu multi-region.

  • In the case of dual-region Cloud Storage buckets, Dataplex creates a publishing dataset in the region corresponding to the region of the lake. When you attach the Cloud Storage bucket as an asset to the lake, Dataplex validates that one of the data locations of the Cloud Storage bucket matches the region of the Dataplex lake.

What's next?