This page describes how to use cross-bucket replication, which uses Storage Transfer Service to copy new and updated objects asynchronously from a source bucket to a destination bucket. When you use cross-bucket replication, you create and manage replication jobs, which are a type of job in Storage Transfer Service.
Before you begin
To use cross-bucket replication, you must complete prerequisite steps. For instructions, refer to Before you begin in the Storage Transfer Service documentation.
Create a replication job
Console
When you create a cross-bucket replication job using the
Google Cloud console, the Storage Transfer Service service agent is
automatically granted the storage.legacyBucketOwner role on the source
bucket and the storage.legacyBucketWriter role on the destination
bucket.
When using the Google Cloud console, you can create a replication job for existing buckets or for new buckets during the bucket creation process.
To create a replication job for a new bucket, follow the instructions for creating a new bucket.
To create a replication job for an existing bucket, complete the following steps:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket whose objects you want to replicate.
On the Bucket details page, click the Configuration tab.
Locate the Cross-bucket replication option and click Edit.
In the Edit cross-bucket replication pane that opens, click Add a destination.
In the Choose a destination section, select a destination bucket, then click Next.
In the Choose replication settings section, do the following:
Optional: To filter objects to replicate by object name prefix, select the Replicate objects based on prefix checkbox in the Choose which objects to replicate section.
To include objects by prefix, enter a prefix in the Include objects with prefix section, then click Add a prefix.
To exclude objects by prefix, enter a prefix in the Exclude objects with prefix section, then click Add a prefix.
Optional: To set a storage class for replicated objects, select a storage class from the menu in the Set storage class for replicated objects section.
If you skip this step, replicated objects use the destination bucket's storage class by default.
- Click Save.
Command line
For Google Cloud CLI instructions on how to create a replication job, refer to Create a replication job using the Google Cloud CLI.
REST APIs
JSON API
For JSON API instructions on how to create a replication job, refer to Create a replication job using the JSON API.
To check the status of the replication job, view Cloud Logging for Storage Transfer Service logs.
List replication jobs
Console
You cannot list replication jobs using the Google Cloud console. See View replication job for instructions on how to view a single replication job at a time.
Command line
For Google Cloud CLI instructions on how to list replication jobs, refer to List replication jobs using the Google Cloud CLI.
REST APIs
JSON API
For JSON API instructions on how to list replication jobs, refer to List replication jobs using the JSON API.
View a replication job
Console
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket whose cross-bucket replication job you want to view.
On the Bucket details page, click the Configuration tab.
Locate the Cross-bucket replication option and click Edit.
The Edit cross-bucket replication pane appears, which displays the replication job for each destination bucket.
On the Buckets page, you can view the Replication column, which displays whether a bucket has a Turbo replication job or a cross-bucket replication job running. For instructions on displaying the Replication column, see Show columns.
Command line
For Google Cloud CLI instructions on how to view a replication job, refer to View a replication job using the Google Cloud CLI.
REST APIs
JSON API
For JSON API instructions on how to view a replication job, refer to View a replication job using the JSON API.
Update a replication job
You can update the following fields of a replication job:
The description of the replication job
The configuration for running a replication job
The configuration of notifications published to Pub/Sub
The logging behavior for replication job operations
The status of the replication job (whether it's enabled, disabled, or deleted)
Console
When using the Google Cloud console, you can only update a replication job by pausing or unpausing the job.
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket that you want to pause or unpause replicating.
On the Bucket details page, click the Configuration tab.
Locate the Cross-bucket replication option and click Edit.
In the Edit cross-bucket replication pane that appears, click Pause or Unpause next to the replication job you want to update.
Command line
For Google Cloud CLI instructions on how to update a replication job, refer to Update a replication job using the Google Cloud CLI.
REST APIs
JSON API
For JSON API instructions on how to update a replication job, refer to Update a replication job using the JSON API.
Delete a replication job
Console
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket you want to stop replicating.
On the Bucket details page, click the Configuration tab.
Locate the Cross-bucket replication option and click Edit.
In the Edit cross-bucket replication pane that appears, click Delete next to the replication job you want to delete.
In the dialogue that appears, click Confirm.
Command line
For Google Cloud CLI instructions on how to delete a replication job, refer to Delete a replication job using the Google Cloud CLI.
REST APIs
JSON API
For JSON API instructions on how to delete a replication job, refer to Delete a replication job using the JSON API.