Regional resources

When you attach a Cloud Storage bucket as a Dataplex Universal Catalog asset, Dataplex Universal Catalog creates a publishing dataset in the corresponding region to publish tables discovered in the bucket.

This page describes how Dataplex Universal Catalog maps single, dual, and multiple regions in Cloud Storage to BigQuery publishing datasets.

Mapping of Cloud Storage regions to BigQuery datasets

Dataplex Universal Catalog lakes, zones, and assets are regional resources, and can reside in one or more regions. BigQuery datasets and Cloud Storage buckets are also regional resources that can reside in one or more regions.

The following are the differences between the regional resources available in Cloud Storage and in BigQuery:

  • Both Cloud Storage and BigQuery support single-region resources.

  • Cloud Storage has dual-regions, whereas BigQuery doesn't.

  • Both Cloud Storage and BigQuery have multi-regions, but they are different.

You can attach Cloud Storage buckets and BigQuery datasets to Dataplex Universal Catalog zones or lakes as Dataplex Universal Catalog assets. Dataplex Universal Catalog automates the creation of publishing datasets for Cloud Storage buckets attached as assets.

Dataplex Universal Catalog ensures that the BigQuery and Cloud Storage regions match. If there is no overlap between your Dataplex Universal Catalog lake's region and one of the Cloud Storage bucket's regions, you can't add the bucket to your lake's zone.

  • In the case of a single-region Cloud Storage bucket, Dataplex Universal Catalog creates a single-region publishing dataset in the same region as the bucket.

  • In the case of the Cloud Storage bucket located in either the Cloud Storage us multi-region or the Cloud Storage eu multi-region, Dataplex Universal Catalog creates a publishing dataset in the corresponding BigQuery us or eu multi-region.

  • In the case of dual-region Cloud Storage buckets, Dataplex Universal Catalog creates a publishing dataset in the region corresponding to the region of the lake. When you attach the Cloud Storage bucket as an asset to the lake, Dataplex Universal Catalog validates that one of the data locations of the Cloud Storage bucket matches the region of the Dataplex Universal Catalog lake.

What's next?