Stay organized with collections
Save and categorize content based on your preferences.
When you create a Dataproc cluster, cluster resources use
a regional endpoints based on
Compute Engine zones.
When you choose a region, you can select a zone within that region, or you can omit
the zone to have the Dataproc Auto Zone feature select a zone for you in
the region you choose. Once a zone is selected, all nodes for that cluster will be
deployed to that zone.
Auto Zone and resource reservations
Auto Zone prioritizes creating a cluster in a zone with
resource reservations, as
follows:
If requested cluster resources can be fully satisfied by reserved, plus,
if necessary, on-demand resources in a zone, Auto Zone will consume the
reserved and on-demand resources, and create the cluster in that zone.
Auto Zone prioritizes zones for selection according to total CPU core (vCPU)
reservations in a zone.
Example: A cluster creation request specifies 20 n2-standard-2 and 1 n2-standard-64
(40 + 64 vCPUs requested). Auto Zone will prioritize the following zones
for selection according to the total vCPU reservations available in the zone:
zone-c available reservations: 3 n2-standard-2 and 1 n2-standard-64 (70 vCPUs)
zone-b available reservations: 1 n2-standard-64 (64 vCPUs)
zone-a available reservations: 25 n2-standard-2 (50 vCPUs)
Assuming each of these zones has additional on-demand vCPU and other
resources sufficient to satisfy the cluster request, Auto Zone will
select zone-c for cluster creation.
If requested cluster resources cannot be fully satisfied by reserved plus
on-demand resources in a zone, Auto Zone will create the cluster in a zone
that is most likely to satisfy the request using on-demand resources.
Use Auto Zone placement
Console
To create a Dataproc cluster that uses Auto Zone placement:
To create a Dataproc cluster that uses Auto Zone placement, use the
gcloud dataproc clusters create
command. Set the --region flag to a region, and
omit the --zone flag (or leave the flag empty: --zone= or zone="").
To create a Dataproc cluster that uses Auto Zone placement,
construct a JSON clusters.create
API request, leaving the
gceClusterConfig.zoneUri
field empty. In the REST endpoint,
https://dataproc.googleapis.com/v1/projects/projectId/regions/region/clusters, insert a region name. Dataproc Auto Zone will choose
a zone for the cluster within the specified region.
Use short resource names with Auto Zone placement: When specifying a resource URI, such as
machineTypeUri or
acceleratorTypeUri, in an Auto Zone placement REST API cluster creation request,
use a short resource name without a zone specification, for example,
"n1-standard-2" or "nvidia-tesla-t4".
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[[["\u003cp\u003eDataproc clusters are deployed to a specific Compute Engine zone within a chosen region, and the Auto Zone feature can automatically select this zone.\u003c/p\u003e\n"],["\u003cp\u003eWhen using Auto Zone, the system prioritizes zones that can fulfill the cluster's resource needs using a combination of reserved and on-demand resources, with a preference for zones that offer higher total vCPU reservations.\u003c/p\u003e\n"],["\u003cp\u003eIf reserved and on-demand resources are insufficient, Auto Zone selects a zone with the best likelihood of satisfying the request using on-demand resources.\u003c/p\u003e\n"],["\u003cp\u003eTo utilize Auto Zone in the Google Cloud console, select a region and choose "Any" under the Zone setting when creating a Dataproc cluster.\u003c/p\u003e\n"],["\u003cp\u003eTo enable Auto Zone with the gcloud command, specify a region and either omit the \u003ccode\u003e--zone\u003c/code\u003e flag or set it to empty (\u003ccode\u003e--zone=""\u003c/code\u003e), and when using the REST API, leave the \u003ccode\u003egceClusterConfig.zoneUri\u003c/code\u003e field empty.\u003c/p\u003e\n"]]],[],null,["# Dataproc Auto Zone placement\n\nWhen you create a Dataproc cluster, cluster resources use\na [regional endpoints](/dataproc/docs/concepts/regional-endpoints) based on\n[Compute Engine zones](/compute/docs/regions-zones/regions-zones#available).\nWhen you choose a region, you can select a zone within that region, or you can omit\nthe zone to have the Dataproc Auto Zone feature select a zone for you in\nthe region you choose. Once a zone is selected, all nodes for that cluster will be\ndeployed to that zone.\n\nAuto Zone and resource reservations\n-----------------------------------\n\nAuto Zone prioritizes creating a cluster in a zone with\n[resource reservations](/compute/docs/instances/choose-reservation-type), as\nfollows:\n\n- If requested cluster resources can be fully satisfied by reserved, plus,\n if necessary, on-demand resources in a zone, Auto Zone will consume the\n reserved and on-demand resources, and create the cluster in that zone.\n\n- Auto Zone prioritizes zones for selection according to total CPU core (`vCPU`)\n reservations in a zone.\n\n **Example:** A cluster creation request specifies 20 `n2-standard-2` and 1 `n2-standard-64`\n (40 + 64 `vCPUs` requested). Auto Zone will prioritize the following zones\n for selection according to the total vCPU reservations available in the zone:\n 1. `zone-c` available reservations: 3 `n2-standard-2` and 1 `n2-standard-64` (70 `vCPUs`)\n 2. `zone-b` available reservations: 1 `n2-standard-64` (64 `vCPUs`)\n 3. `zone-a` available reservations: 25 `n2-standard-2` (50 `vCPUs`)\n\n Assuming each of these zones has additional on-demand `vCPU` and other\n resources sufficient to satisfy the cluster request, Auto Zone will\n select `zone-c` for cluster creation.\n- If requested cluster resources cannot be fully satisfied by reserved plus\n on-demand resources in a zone, Auto Zone will create the cluster in a zone\n that is most likely to satisfy the request using on-demand resources.\n\nUse Auto Zone placement\n-----------------------\n\n### Console\n\nTo create a Dataproc cluster that uses Auto Zone placement:\n\n1. In the Google Cloud console, open the Dataproc [Create a Dataproc cluster on Compute Engine](https://console.cloud.google.com/dataproc/clustersAdd) page. The **Set up cluster** panel is selected.\n2. In the **Location** section, do the following:\n - Select a **Region** for your cluster.\n - Under **Zone**, select \"Any\".\n\n### gcloud command\n\nTo create a Dataproc cluster that uses Auto Zone placement, use the\n[`gcloud dataproc clusters create`](/sdk/gcloud/reference/dataproc/clusters/create)\ncommand. Set the `--region` flag to a region, and\nomit the `--zone` flag (or leave the flag empty: `--zone=` or `zone=\"\"`).\n**Note:** If you ran the `gcloud config set compute/`\u003cvar translate=\"no\"\u003ezone\u003c/var\u003e command to set a default zone, you must supply the `--zone=` or `zone=\"\"` flag to `gcloud dataproc clusters create` in order to enable the Auto Zone feature. [Power Shell](https://docs.microsoft.com/en-us/powershell/) users must surround the empty quotes with single quotes: `--zone='\"\"'`. \n\n```\ngcloud dataproc clusters create cluster-name \\\n --region=region \\\n --zone=\"\" \\\n other args ...\n```\n\n### REST API\n\nTo create a Dataproc cluster that uses Auto Zone placement,\nconstruct a JSON [clusters.create](/dataproc/docs/reference/rest/v1/projects.regions.clusters/create)\nAPI request, leaving the\n[gceClusterConfig.zoneUri](/dataproc/docs/reference/rest/v1/ClusterConfig#GceClusterConfig)\nfield empty. In the REST endpoint,\n`https://dataproc.googleapis.com/v1/projects/`\u003cvar translate=\"no\"\u003eprojectId\u003c/var\u003e`/regions/`\u003cvar translate=\"no\"\u003eregion\u003c/var\u003e`/clusters`, insert a region name. Dataproc Auto Zone will choose\na zone for the cluster within the specified region.\n\n**Use short resource names with Auto Zone placement** : When specifying a resource URI, such as\n[machineTypeUri](/dataproc/docs/reference/rest/v1/ClusterConfig#InstanceGroupConfig) or\n[acceleratorTypeUri](/dataproc/docs/reference/rest/v1/ClusterConfig#acceleratorconfig), in an Auto Zone placement REST API cluster creation request,\nuse a short resource name without a zone specification, for example,\n\"n1-standard-2\" or \"nvidia-tesla-t4\"."]]
