Hostname of the first master node. The value is either [CLUSTER_NAME]-m in a standard or single node cluster, or [CLUSTER_NAME]-m-0 in a high-availability cluster, where [CLUSTER_NAME] is the name of your cluster.
dataproc-master-additional
Comma-separated list of hostnames for the additional master nodes in a high-availability cluster, for example, [CLUSTER_NAME]-m-1,[CLUSTER_NAME]-m-2 in a cluster that has 3 master nodes.
SPARK_BQ_CONNECTOR_VERSION or SPARK_BQ_CONNECTOR_URL
The version or URL that points to a Spark BigQuery connector version to use in Spark applications, for example, 0.42.1 or gs://spark-lib/bigquery/spark-3.5-bigquery-0.42.1.jar. A default Spark BigQuery connector version is pre-installed in Dataproc 2.1 and later image version clusters. For more information, see Use the Spark BigQuery connector.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[[["\u003cp\u003eCustom metadata is accessible to processes within your cluster and can be utilized by initialization actions.\u003c/p\u003e\n"],["\u003cp\u003eLabels, while not directly available to cluster processes, are used for searching resources via the Dataproc API.\u003c/p\u003e\n"],["\u003cp\u003eIf data needs to be accessible to the cluster and also be a search parameter, it should be added as both metadata and a label.\u003c/p\u003e\n"],["\u003cp\u003eDataproc provides predefined metadata keys like \u003ccode\u003edataproc-bucket\u003c/code\u003e, \u003ccode\u003edataproc-region\u003c/code\u003e, \u003ccode\u003edataproc-worker-count\u003c/code\u003e, and others, to manage cluster operations.\u003c/p\u003e\n"],["\u003cp\u003eCustom metadata can be set during cluster creation using the \u003ccode\u003e--metadata\u003c/code\u003e flag with the gcloud CLI's \u003ccode\u003egcloud dataproc clusters create\u003c/code\u003e command.\u003c/p\u003e\n"]]],[],null,["# Cluster metadata\n\n| **Metadata compared to Labels**\n|\n| - Custom metadata is available to processes running on your cluster, and can be used by initialization actions.\n| - Labels are not readily available to processes running on your cluster, but can be used when searching through resources with the Dataproc API.\n| If you need a piece of data to be available to your cluster and also used as an API search parameter, then add it both as metadata and as a label to your cluster.\n\nDataproc sets special metadata values for the instances that run in your\ncluster:\n\nYou can use these values to customize the behavior of\n[initialization actions](/dataproc/docs/concepts/configuring-clusters/init-actions).\n\nYou can use the `--metadata` flag in the\n[gcloud dataproc clusters create](/sdk/gcloud/reference/dataproc/clusters/create)\ncommand to provide your own metadata: \n\n```\ngcloud dataproc clusters create CLUSTER_NAME \\\n --region=REGION \\\n --metadata=name1=value1,name2=value2... \\\n ... other flags ...\n```"]]
