Stay organized with collections
Save and categorize content based on your preferences.
This document shows you how run a standalone tool in preparation for an
upgrade. Before upgrading an admin or user cluster that is at Google Distributed Cloud
version 1.9 and later versions, we recommend that you run the pre-upgrade tool.
To run the tool, use the bash script in this document that uses a
hard-coded Secure Hash Algorithms (SHA). For each release of the tool, this
document will be updated with the new SHA. The script creates a Kubernetes Job
to run a specific version of preflight checks depending on the version that you
are upgrading to.
Pre-upgrade checks
The tool checks the following before you upgrade a user cluster:
Category
Description
Cluster Health
Validates the PDBs in all namespaces of the admin cluster.
Validates the PDBs in all namespaces of the user cluster.
Validates that a previous upgrade of the admin cluster finished successfully.
Configurations
Recommends the patch version to upgrade to.
Checks whether the component access SA key is wiped out as
described in the related
known issue.
If you are upgrading to 1.10, warns if you will need to apply the
workaround in this known
issue.
The tool checks the following before you upgrade an admin cluster:
Category
Description
Cluster Health
Validates the PodDisruptionBudgets (PDBs) in all
namespaces of the admin cluster.
TARGET_VERSION: The
Google Distributed Cloud patch version
that you want to upgrade to. The format for the version number must be
a complete patch version, like 1.13.10-gke.42.
ADMIN_CLUSTER_KUBECONFIG: The path to the admin
cluster kubeconfig.
If you are using a private registry, download the preflight container
image with the provided docker digest and upload the image into the private
registry. If you aren't using a private registry, skip to the next step.
Replace REGISTRY_ADDRESS with the private registry
address.
In the following bash script, set values for these placeholders:
ADMIN_CLUSTER_KUBECONFIG: The path to the admin
cluster kubeconfig.
REGISTRY_ADDRESS: If the admin cluster uses a
private registry, this is the private registry address that you specified
in the previous step. If you aren't using a private registry, specify
the public registry: gcr.io/gke-on-prem-release
JOB_NAME: This is the name of the job that the script
outputs from the previous step.
JOB_NAMESPACE: The value that you set depends on
whether you are upgrading an admin or user cluster. If you are
upgrading an admin cluster, specify kube-system. If you are
upgrading a user cluster, specify
USER_CLUSTER_NAME-gke-onprem-mgmt.
Wait a few minutes for the job to complete or to reach the
backoff limit and fail. In the results, review the Reason for any check
with the status Warning, Unknown or Failure to see if you can resolve
the issue.
Before upgrading clusters, run the following command to delete the Job:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[],[],null,["# Run the pre-upgrade tool\n\nThis document shows you how run a standalone tool in preparation for an\nupgrade. Before upgrading an admin or user cluster that is at Google Distributed Cloud\n**version 1.9 and later versions**, we recommend that you run the pre-upgrade tool.\n\nTo run the tool, use the bash script in this document that uses a\nhard-coded Secure Hash Algorithms (SHA). For each release of the tool, this\ndocument will be updated with the new SHA. The script creates a Kubernetes Job\nto run a specific version of preflight checks depending on the version that you\nare upgrading to.\n\nPre-upgrade checks\n------------------\n\nThe tool checks the following before you upgrade a user cluster:\n\nThe tool checks the following before you upgrade an admin cluster:\n\nPrepare to run the tool\n-----------------------\n\n1. [Upgrade your admin workstation](/kubernetes-engine/distributed-cloud/vmware/docs/how-to/upgrading#upgrade_awstation)\n if you have not done so.\n\n2. Run `gkectl prepare` to import OS images to vSphere if you have not done so:\n\n gkectl prepare \\\n --bundle-path /var/lib/gke/bundles/gke-onprem-vsphere-\u003cvar translate=\"no\"\u003eTARGET_VERSION\u003c/var\u003e.tgz \\\n --kubeconfig \u003cvar translate=\"no\"\u003eADMIN_CLUSTER_KUBECONFIG\u003c/var\u003e\n\n Replace the following:\n - \u003cvar translate=\"no\"\u003eTARGET_VERSION\u003c/var\u003e: The\n [Google Distributed Cloud patch version](/kubernetes-engine/distributed-cloud/vmware/docs/version-history#supported-versions)\n that you want to upgrade to. The format for the version number must be\n a complete patch version, like `1.13.10-gke.42`.\n\n - \u003cvar translate=\"no\"\u003eADMIN_CLUSTER_KUBECONFIG\u003c/var\u003e: The path to the admin\n cluster kubeconfig.\n\n | **Note:** You don't need to re-run `gkectl prepare` when you are ready to upgrade the cluster.\n3. If you are using a private registry, download the preflight container\n image with the provided docker digest and upload the image into the private\n registry. If you aren't using a private registry, skip to the next step.\n\n export SRC_IMAGE=gcr.io/gke-on-prem-release/preflight@sha256:9704315c6637750a014d0079ca04a8f97d0ca3735e175020377107c3181f6234\n export DST_IMAGE=\u003cvar translate=\"no\"\u003eREGISTRY_ADDRESS\u003c/var\u003e/preflight:$(date +%Y-%m%d-%H%M%S)\n docker pull $SRC_IMAGE\n docker tag $SRC_IMAGE $DST_IMAGE\n docker push $DST_IMAGE\n\n Replace \u003cvar translate=\"no\"\u003eREGISTRY_ADDRESS\u003c/var\u003e with the private registry\n address.\n4. In the following bash script, set values for these placeholders:\n\n - \u003cvar translate=\"no\"\u003eADMIN_CLUSTER_KUBECONFIG\u003c/var\u003e: The path to the admin\n cluster kubeconfig.\n\n - \u003cvar translate=\"no\"\u003eREGISTRY_ADDRESS\u003c/var\u003e: If the admin cluster uses a\n private registry, this is the private registry address that you specified\n in the previous step. If you aren't using a private registry, specify\n the public registry: `gcr.io/gke-on-prem-release`\n\n #!/bin/bash\n UPGRADE_TARGET_VERSION=${1}\n CLUSTER_NAME=${2}\n ADMIN_KUBECONFIG=\u003cvar translate=\"no\"\u003eADMIN_CLUSTER_KUBECONFIG\u003c/var\u003e\n REGISTRY_ADDRESS=\u003cvar translate=\"no\"\u003eREGISTRY_ADDRESS\u003c/var\u003e\n pre_upgrade_namespace=kube-system\n if [[ -z \"$CLUSTER_NAME\" ]]\n then\n echo \"Running the pre-ugprade tool before admin cluster upgrade\"\n else\n echo \"Running the pre-ugprade tool before user cluster upgrade\"\n pre_upgrade_namespace=$CLUSTER_NAME-gke-onprem-mgmt\n fi\n kubectl apply --kubeconfig ${ADMIN_KUBECONFIG} -f - \u003c\u003cEOF\n apiVersion: v1\n kind: ServiceAccount\n metadata:\n name: pre-upgrade-job\n namespace: $pre_upgrade_namespace\n EOF\n kubectl apply --kubeconfig ${ADMIN_KUBECONFIG} -f - \u003c\u003cEOF\n apiVersion: rbac.authorization.k8s.io/v1\n kind: ClusterRoleBinding\n metadata:\n creationTimestamp: null\n name: pre-upgrade-job-rolebinding-in-$pre_upgrade_namespace\n roleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: onprem-user-cluster-controller-role\n subjects:\n - kind: ServiceAccount\n name: pre-upgrade-job\n namespace: $pre_upgrade_namespace\n EOF\n kubectl apply --kubeconfig ${ADMIN_KUBECONFIG} -f - \u003c\u003cEOF\n apiVersion: batch/v1\n kind: Job\n metadata:\n name: pre-upgrade-$(date +%Y-%m%d-%H%M%S)\n namespace: $pre_upgrade_namespace\n labels:\n onprem.cluster.gke.io/job-usage: preflight\n spec:\n ttlSecondsAfterFinished: 2592000\n backoffLimit: 2\n template:\n metadata:\n labels:\n onprem.cluster.gke.io/pod-usage: preflight\n spec:\n containers:\n - name: preflight\n image: $REGISTRY_ADDRESS/preflight@sha256:9704315c6637750a014d0079ca04a8f97d0ca3735e175020377107c3181f6234\n imagePullPolicy: Always\n command:\n - /preflight\n - --upgrade-target-version\n - \"$UPGRADE_TARGET_VERSION\"\n - --cluster-name\n - \"$CLUSTER_NAME\"\n - --scenario\n - pre-upgrade\n restartPolicy: Never\n serviceAccountName: pre-upgrade-job\n imagePullSecrets:\n - name: private-registry-creds\n EOF\n\n5. Save the above bash script to a file called `pre-upgrade.sh` and make\n it executable:\n\n chmod +x pre-upgrade.sh\n\nRun the script\n--------------\n\n1. The arguments that you provide when you run the script depends on whether\n you are upgrading a user cluster or an admin cluster:\n\n - Before upgrading an admin cluster run the script as follows:\n\n ./pre-upgrade.sh \u003cvar translate=\"no\"\u003eTARGET_VERSION\u003c/var\u003e\n\n - Before upgrading a user cluster:\n\n ./pre-upgrade.sh \u003cvar translate=\"no\"\u003eTARGET_VERSION\u003c/var\u003e \u003cvar translate=\"no\"\u003eUSER_CLUSTER_NAME\u003c/var\u003e\n\n Replace \u003cvar translate=\"no\"\u003eUSER_CLUSTER_NAME\u003c/var\u003e with the name of the user\n cluster you will be upgrading.\n\n The output is similar to the following: \n\n job.batch/pre-upgrade-2023-0822-213551 created\n\n2. Run the following command on the Pods controlled by the job to get\n a list of validation results.\n\n kubectl logs -n \u003cvar translate=\"no\"\u003eJOB_NAMESPACE\u003c/var\u003e jobs/\u003cvar translate=\"no\"\u003eJOB_NAME\u003c/var\u003e \\\n --kubeconfig \u003cvar translate=\"no\"\u003eADMIN_CLUSTER_KUBECONFIG\u003c/var\u003e\n\n Replace the following:\n - \u003cvar translate=\"no\"\u003eJOB_NAME\u003c/var\u003e: This is the name of the job that the script\n outputs from the previous step.\n\n - \u003cvar translate=\"no\"\u003eJOB_NAMESPACE\u003c/var\u003e: The value that you set depends on\n whether you are upgrading an admin or user cluster. If you are\n upgrading an admin cluster, specify `kube-system`. If you are\n upgrading a user cluster, specify\n \u003cvar translate=\"no\"\u003eUSER_CLUSTER_NAME\u003c/var\u003e`-gke-onprem-mgmt`.\n\n Wait a few minutes for the job to complete or to reach the\n backoff limit and fail. In the results, review the `Reason` for any check\n with the status `Warning`, `Unknown` or `Failure` to see if you can resolve\n the issue.\n | **Important:** We recommend that you resolve all issues with a `Failure` status because these issues will cause the upgrade to fail. Contact Google support if needed.\n3. Before upgrading clusters, run the following command to delete the Job:\n\n kubectl delete jobs \u003cvar translate=\"no\"\u003eJOB_NAME\u003c/var\u003e -n \u003cvar translate=\"no\"\u003eJOB_NAMESPACE\u003c/var\u003e \\\n --kubeconfig \u003cvar translate=\"no\"\u003eADMIN_CLUSTER_KUBECONFIG\u003c/var\u003e"]]