Stay organized with collections
Save and categorize content based on your preferences.
Overview
When you create a Dataproc cluster, the Apache Hive application and its
components, including Hive metastore, are installed on the cluster, and
a default password is set in the hive-site.xml file located on the
cluster master node.
Specifying your own Hive metastore password is recommended to accomplish the
following goals:
As a best-practice security measure to ensure you control access to the
local Hive metastore by providing your own password
To specify a known password that controls access to external
Hive metastores used with external databases that are shared among different
clusters
Dataproc does not support the following Hive metastore scenarios,
regardless of whether you use the default or a user-supplied Hive metastore password:
You use an embedded metastore client in the Spark driver running in cluster mode,
so that the worker nodes require Hive passwords. This scenario can cause
connectivity problems with the metastore database since the connection
is not made through the HiveMetaStore process running on the
Dataproc master node.
You deactivate Hive metastore and hive-server2 to use your own MySQL database.
In this scenario, the
spark.hadoop.javax.jdo.option.ConnectionURL=jdbc:mysql://CLUSTER_NAME-m/metastore
property has no effect.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[[["\u003cp\u003eDataproc clusters install Hive, including its metastore, with a default password, which is modifiable during cluster creation.\u003c/p\u003e\n"],["\u003cp\u003eSpecifying a custom Hive metastore password enhances security and controls access to both local and external Hive metastores shared across clusters.\u003c/p\u003e\n"],["\u003cp\u003eYou can set the Hive metastore password during cluster creation using the \u003ccode\u003egcloud dataproc clusters create\u003c/code\u003e command with the \u003ccode\u003ehive:javax.jdo.option.ConnectionPassword\u003c/code\u003e property.\u003c/p\u003e\n"],["\u003cp\u003eThe custom Hive metastore password is only stored on the master node(s) of the Dataproc cluster, not on worker nodes.\u003c/p\u003e\n"],["\u003cp\u003eDataproc does not support embedded metastore clients in the Spark driver or deactivation of the hive metastore to use a custom MySQL database.\u003c/p\u003e\n"]]],[],null,["# Create and secure a Hive metastore cluster\n\n| **Objective:** Replace the default Hive metastore password when you create a Dataproc cluster for added security.\n\nOverview\n--------\n\nWhen you create a Dataproc cluster, the Apache Hive application and its\ncomponents, including Hive metastore, are installed on the cluster, and\na default password is set in the `hive-site.xml` file located on the\ncluster master node.\n\nSpecifying your own Hive metastore password is recommended to accomplish the\nfollowing goals:\n\n- As a best-practice security measure to ensure you control access to the\n local Hive metastore by providing your own password\n\n- To specify a known password that controls access to external\n Hive metastores used with external databases that are shared among different\n clusters\n\nSet the Hive metastore password\n-------------------------------\n\nRun the following\n[Google Cloud CLI Dataproc clusters create](/sdk/gcloud/reference/dataproc/clusters/create)\ncommand to create a Dataproc cluster and specify a Hive\nmetastore password. \n\n```\ngcloud dataproc clusters create cluster-name\n --properties=\"hive:javax.jdo.option.ConnectionPassword=HIVE_METASTORE_PASSWORD\"\n```\n\nNotes:\n\n- See [Create a key](/kms/docs/create-key) to create a key in Cloud Key Management Service.\n- The Hive metastore password is stored only on cluster master node(s), not on worker nodes.\n\nFor additional information on securing Dataproc clusters see\n[Dataproc security best practices](/dataproc/docs/concepts/security-best-practices).\n\nUnsupported Scenarios\n---------------------\n\nDataproc does not support the following Hive metastore scenarios,\nregardless of whether you use the default or a user-supplied Hive metastore password:\n\n- You use an embedded metastore client in the Spark driver running in cluster mode,\n so that the worker nodes require Hive passwords. This scenario can cause\n connectivity problems with the metastore database since the connection\n is not made through the `HiveMetaStore` process running on the\n Dataproc master node.\n\n- You deactivate Hive metastore and `hive-server2` to use your own MySQL database.\n In this scenario, the\n `spark.hadoop.javax.jdo.option.ConnectionURL=jdbc:mysql://`\u003cvar translate=\"no\"\u003eCLUSTER_NAME\u003c/var\u003e`-m/metastore`\n property has no effect."]]
