Membuat ulang dan mengupdate Dataproc di cluster virtual GKE
Tetap teratur dengan koleksi
Simpan dan kategorikan konten berdasarkan preferensi Anda.
Anda dapat menyalin konfigurasi cluster virtual Dataproc di GKE yang ada,
memperbarui konfigurasi yang disalin, lalu membuat cluster Dataproc di GKE baru menggunakan konfigurasi yang diperbarui.
Membuat ulang dan mengupdate cluster Dataproc di GKE
gcloud
Menetapkan variabel lingkungan:
CLUSTER=existing Dataproc on GKE cluster name \
REGION=region
Ekspor konfigurasi cluster Dataproc di GKE yang ada ke file YAML.
Tunggu hingga operasi penghapusan sebelumnya selesai, lalu impor konfigurasi cluster yang telah diupdate untuk membuat cluster virtual Dataproc di GKE baru dengan setelan konfigurasi yang telah diupdate.
Lakukan perubahan tambahan untuk memperbarui setelan konfigurasi cluster virtual Dataproc di GKE, seperti mengubah componentVersion Spark.
Hapus cluster virtual Dataproc on GKE yang ada jika Anda akan membuat cluster
yang memiliki nama yang sama dengan cluster yang sedang diupdate (jika Anda mengganti
cluster asli).
Tunggu hingga operasi penghapusan sebelumnya selesai, lalu impor konfigurasi cluster yang telah diupdate untuk membuat cluster virtual Dataproc di GKE baru dengan setelan yang telah diupdate.
[[["Mudah dipahami","easyToUnderstand","thumb-up"],["Memecahkan masalah saya","solvedMyProblem","thumb-up"],["Lainnya","otherUp","thumb-up"]],[["Sulit dipahami","hardToUnderstand","thumb-down"],["Informasi atau kode contoh salah","incorrectInformationOrSampleCode","thumb-down"],["Informasi/contoh yang saya butuhkan tidak ada","missingTheInformationSamplesINeed","thumb-down"],["Masalah terjemahan","translationIssue","thumb-down"],["Lainnya","otherDown","thumb-down"]],["Terakhir diperbarui pada 2025-09-02 UTC."],[[["\u003cp\u003eYou can recreate a Dataproc on GKE cluster by copying an existing cluster's configuration, updating it, and then creating a new cluster with the updated settings.\u003c/p\u003e\n"],["\u003cp\u003eWhen updating the configuration, the \u003ccode\u003ekubernetesNamespace\u003c/code\u003e field must be removed to prevent namespace conflicts.\u003c/p\u003e\n"],["\u003cp\u003eIf the new cluster will have the same name as the existing one, the original cluster should be deleted prior to the creation of the new one.\u003c/p\u003e\n"],["\u003cp\u003eThe update process includes the option to modify settings such as the Spark component version within the cluster's configuration.\u003c/p\u003e\n"],["\u003cp\u003eThe Google Cloud Console does not support the functionality to recreate a Dataproc on GKE virtual cluster by importing an existing configuration.\u003c/p\u003e\n"]]],[],null,["# Recreate and update a Dataproc on GKE virtual cluster\n\nYou can copy an existing Dataproc on GKE virtual cluster's configuration,\nupdate the copied configuration, and then create a new Dataproc on GKE\ncluster using the updated configuration.\n\nRecreate and update a Dataproc on GKE cluster\n---------------------------------------------\n\n### gcloud\n\n1. Set environment variables:\n\n ```\n CLUSTER=existing Dataproc on GKE cluster name \\\n REGION=/compute/docs/regions-zones#available\n ```\n\n \u003cbr /\u003e\n\n2. Export the existing Dataproc on GKE cluster configuration to a YAML file.\n\n ```\n gcloud dataproc clusters export $CLUSTER \\\n --region=$REGION \u003e \"${CLUSTER}-config.yaml\"\n ```\n\n \u003cbr /\u003e\n\n3. Update the configuration.\n\n 1. Remove the\n [`kubernetesNamespace`](/dataproc/docs/reference/rest/v1/projects.regions.clusters#KubernetesClusterConfig.FIELDS.kubernetes_namespace)\n field. Removing this field is necessary to avoid a namespace conflict\n when you create the updated cluster.\n\n Sample `sed` command to remove the `kubernetesNamespace` field: \n\n ```\n sed -E \"s/kubernetesNamespace: .+$//g\" ${CLUSTER}-config.yaml\n ```\n\n \u003cbr /\u003e\n\n 2. Make additional changes to update Dataproc on GKE virtual cluster\n configuration settings, such as changing the Spark\n [componentVersion](/dataproc/docs/reference/rest/v1/projects.regions.clusters#KubernetesSoftwareConfig.FIELDS.component_version).\n\n4. [Delete the existing Dataproc on GKE virtual cluster](/dataproc/docs/guides/dpgke/dataproc-gke-delete-cluster) if you will create a cluster that\n has the same name as the cluster it is updating (if you are replacing the\n original cluster).\n\n5. Wait for the previous delete operation to finish, and then import the\n updated cluster configuration to create a new Dataproc on GKE\n virtual cluster with the updated config settings.\n\n ```\n gcloud dataproc clusters import $CLUSTER \\\n --region=$REGION \\\n --source=\"${CLUSTER}-config.yaml\"\n ```\n\n### API\n\n1. Set environment variables:\n\n ```\n CLUSTER=existing Dataproc on GKE cluster name \\\n REGION=/compute/docs/regions-zones#available\n ```\n\n \u003cbr /\u003e\n\n2. Export the existing Dataproc on GKE cluster configuration to a YAML file.\n\n ```\n curl -X GET -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \"https://dataproc.googleapis.com/v1/projects/${PROJECT}/regions/${REGION}/clusters/${CLUSTER}?alt=json\" \u003e \"${CLUSTER}-config.json\"\n ```\n\n \u003cbr /\u003e\n\n3. Update the configuration.\n\n 1. Remove the\n [`kubernetesNamespace`](/dataproc/docs/reference/rest/v1/projects.regions.clusters#KubernetesClusterConfig.FIELDS.kubernetes_namespace)\n field. Removal of this field is necessary to avoid a namespace conflict\n when you create the updated cluster.\n\n Sample `jq` command to remove `kubernetesNamespace` field: \n\n ```\n jq 'del(.virtualClusterConfig.kubernetesClusterConfig.kubernetesNamespace)'\n ```\n\n \u003cbr /\u003e\n\n 2. Make additional changes to update Dataproc on GKE virtual cluster\n configuration settings, such as changing the Spark\n [componentVersion](/dataproc/docs/reference/rest/v1/projects.regions.clusters#KubernetesSoftwareConfig.FIELDS.component_version).\n\n4. Delete the existing Dataproc on GKE virtual cluster if you will create a cluster\n that has the same name as the cluster it is updating (if you are replacing the\n original cluster).\n\n ```\n curl -X DELETE -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \"https://dataproc.googleapis.com/v1/projects/${PROJECT}/regions/${REGION}/clusters/${CLUSTER}\"\n ```\n\n \u003cbr /\u003e\n\n5. Wait for the previous delete operation to finish, and\n then import the updated cluster configuration to create a new Dataproc on GKE\n virtual cluster with the updated settings.\n\n ```\n curl -i -X POST -H \"Authorization: Bearer $(gcloud auth print-access-token)\" -H \"Content-Type: application/json; charset=utf-8\" -d \"@${CLUSTER}-config.json\" \"https://dataproc.googleapis.com/v1/projects/${PROJECT}/regions/${REGION}/clusters?alt=json\"\n ```\n\n### Console\n\nThe Google Cloud console does not support recreating a Dataproc on GKE\nvirtual cluster by importing an existing cluster's configuration.\n\n\u003cbr /\u003e"]]
