[[["易于理解","easyToUnderstand","thumb-up"],["解决了我的问题","solvedMyProblem","thumb-up"],["其他","otherUp","thumb-up"]],[["很难理解","hardToUnderstand","thumb-down"],["信息或示例代码不正确","incorrectInformationOrSampleCode","thumb-down"],["没有我需要的信息/示例","missingTheInformationSamplesINeed","thumb-down"],["翻译问题","translationIssue","thumb-down"],["其他","otherDown","thumb-down"]],["最后更新时间 (UTC)：2025-09-04。"],[[["\u003cp\u003eDataproc Serverless can use Secret Manager as a credential store to securely manage sensitive data like API keys, passwords, and certificates.\u003c/p\u003e\n"],["\u003cp\u003eTo utilize Secret Manager, configure the \u003ccode\u003ehadoop.security.credential.provider.path\u003c/code\u003e property with the \u003ccode\u003egsm://\u003c/code\u003e scheme when submitting a Dataproc Serverless workload.\u003c/p\u003e\n"],["\u003cp\u003eThe \u003ccode\u003ehadoop.security.credstore.google-secret-manager.secret-id.substitute-dot-operator\u003c/code\u003e property allows for the substitution of dots with hyphens in credential names, enabling compatibility with OSS components that use dots.\u003c/p\u003e\n"],["\u003cp\u003eYou can specify a particular secret version with the property \u003ccode\u003ehadoop.security.credstore.google-secret-manager.secret-version\u003c/code\u003e for consistent access, or omit it to use the latest version of the secret.\u003c/p\u003e\n"],["\u003cp\u003eThis feature is available for Dataproc Serverless for Spark runtime versions 1.2.29+, 2.2.29+, or later.\u003c/p\u003e\n"]]],[],null,["# Secret Manager Credential Provider\n\nThis document describes how to use Secret Manager\nas a credential store with Google Cloud Serverless for Apache Spark to safely store and access sensitive\ndata processed by serverless workloads.\n\nOverview\n--------\n\nThe [Secret Manager](/secret-manager/docs/overview) can\nsafeguard your sensitive data, such as your API keys, passwords, and\ncertificates. You can use it to manage, access, and audit your secrets across\nGoogle Cloud.\n\nWhen you run a Serverless for Apache Spark batch workload, you can configure it\nto use a Secret Manager secret by using the\nDataproc Secret Manager Credential Provider.\n\nAvailability\n------------\n\nThis feature is available for Serverless for Apache Spark runtime versions\n1.2.29+, 2.2.29+, or later major\n[runtime versions](/dataproc-serverless/docs/concepts/versions/dataproc-serverless-versions#supported-dataproc-serverless-for-spark-runtime-versions).\n| **Note:** This feature is also available for use with Dataproc on clusters created with image versions 2.0.97+, 2.1.41+, 2.2.6+, or later major [image versions](/dataproc/docs/concepts/versioning/dataproc-version-clusters#supported-dataproc-image-versions). For Dataproc on Compute Engine information, see [Secret Manager Credential Provider for Dataproc](/dataproc/docs/guides/hadoop-google-secret-manager-credential-provider).\n\nTerminology\n-----------\n\nThe following table describes the terms used in this document.\n\nPermissions\n-----------\n\nDataproc checks if the following optional secrets exist:\n\n- fs-gs-encryption-key\n- fs-gs-encryption-key-hash\n- fs-gs-proxy-password\n- fs-gs-proxy-username\n\nTo make sure that the [Dataproc VM Service Account](/dataproc/docs/concepts/iam/dataproc-principals#vm_service_account_data_plane_identity)\nhas permission to check the `fs-gs` secrets, add the\n[Secret Manager Secret Accessor role](/secret-manager/docs/access-control#secretmanager.secretAccessor)\nwith the following condition to the service account, as follows: \n\n```\n{\n \"expression\": \"resource.name.startsWith(\\\"projects/PROJECT_NUMBER/secrets/fs-gs-\\\")\",\n \"title\": \"gsmkeycondition\",\n \"description\": \"Permission to access Dataproc secrets\"\n}\n```\n\nUsage\n-----\n\nYou can configure [supported Hadoop and other OSS components](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html#Usage_Overview)\nto work with the Secret Manager by setting the following\nproperties when you submit a Serverless for Apache Spark workload:\n\n- **Provider path** (required): The provider path property, `spark.hadoop.hadoop.security.credential.provider.path`,\n is a comma-separated list of one or more credential provider URIs that is traversed to resolve a credential.\n\n ```\n --properties=spark.hadoop.hadoop.security.credential.provider.path=gsm://projects/PROJECT_ID\n ```\n - The `scheme` in the provider path indicates the credential provider type. Hadoop schemes include `jceks://`, `user://`,`localjceks://`. Use the `gsm://` scheme to search for credentials in Secret Manager.\n- **Substitute dot operator** : The Secret Manager service does not allow dots (`.`) in secret names. However, some open source software (OSS) components use dots in their credential keys. To fix this limitation, enable this property to replace dots (`.`) with hyphens (`-`) in credential names. This ensures that OSS credentials with dots in their names can be stored and retrieved correctly from Secret Manager.\n\n For example,\n If an OSS credential key is `a.b.c`, you must modify it to `a-b-c` when storing it in Secret Manager. \n\n ```\n --properties=spark.hadoop.hadoop.security.credstore.google-secret-manager.secret-id.substitute-dot-operator=true\n ```\n\n This is an optional property. By default, the value is `false`. For credentials keys that have no dot (`.`) operator in their credential name, this property can be safely ignored.\n- **Secret version** : Secrets in Secret Manager can have multiple versions (values). Use this property to access a specific secret version for stable access in production environments.\n\n ```\n --properties=spark.hadoop.hadoop.security.credstore.google-secret-manager.secret-version=1\n ```\n\n This is an optional property. By default, Secret Manager\n accesses the `LATEST` version, which resolves to the latest value of the secret at runtime. If your use case is to always access the `LATEST` version of a secret, this property can be safely ignored.\n\n### Run a batch workload with Secret Manager Credential Provider\n\nTo [submit a batch workload](/dataproc-serverless/docs/quickstarts/spark-batch#submit_a_spark_batch_workload)\nthat uses Secret Manager Credential Provider, run the following command\nlocally or in [Cloud Shell](/shell). \n\n```\ngcloud dataproc batches submit spark \\\n --region=REGION \\\n --jars=JARS \\\n --class=MAIN_CLASS \\\n --properties=\"spark.hadoop.hadoop.security.credential.provider.path=gsm://projects/PROJECT_ID,spark.hadoop.hadoop.security.credstore.google-secret-manager.secret-id.substitute-dot-operator=true\" \\\n ...other flags as needed...\n```\n\nReplace the following:\n\n- \u003cvar translate=\"no\"\u003eREGION\u003c/var\u003e: a [Compute Engine region](/compute/docs/regions-zones#available) where your workload runs\n- \u003cvar translate=\"no\"\u003eJARS\u003c/var\u003e: workload jar path\n- \u003cvar translate=\"no\"\u003eMAIN_CLASS\u003c/var\u003e: the Jar main class\n- \u003cvar translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e: your project ID, listed in the **Project info** section of the [Google Cloud console dashboard](https://console.cloud.google.com/home/dashboard)"]]
