Stay organized with collections
Save and categorize content based on your preferences.
This document provides information about Dataproc Serverless staging buckets.
Dataproc Serverless creates a Cloud Storage staging bucket in your project
or reuses an existing staging bucket from previous batch
creation requests. This is the default bucket created by
Dataproc on Compute Engine clusters. For more
information, see
Dataproc staging and temp buckets.
Dataproc Serverless stores workload dependencies, config files, and
job driver console output in the staging bucket.
Dataproc Serverless sets regional staging buckets in
Cloud Storage locations
according to the Compute Engine zone where your workload is deployed,
and then creates and manages these project-level, per-location buckets.
Dataproc Serverless-created staging buckets are shared among
workloads in the same region, and are created with a
Cloud Storage soft delete retention
duration set to 0 seconds.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-03-21 UTC."],[[["Dataproc Serverless utilizes Cloud Storage staging buckets to store workload dependencies, config files, and job driver console output."],["These staging buckets are created by Dataproc Serverless within your project, or an existing one is reused, similar to the default bucket used with Dataproc on Compute Engine clusters."],["Dataproc Serverless creates regional staging buckets, which are shared across workloads within the same region, based on the Compute Engine zone where the workload is deployed."],["The staging buckets created by Dataproc Serverless can be identified by filtering for the `dataproc-staging-` prefix in Cloud Storage, and they are created with a 0-second soft delete retention."]]],[]]
