Stay organized with collections
Save and categorize content based on your preferences.
You can reboot any persistent resource that's in the RUNNING or ERROR state.
Rebooting a persistent resource lets you recover from errors that the persistent
resource can't recover from on its own. You can also reboot a persistent
resource to manually obtain more up-to-date clusters. This page shows you how to
reboot a persistent resource by using the Google Cloud console and the REST API.
Select one of the following tabs for instructions on how to reboot a persistent
resource. Make sure there's no training jobs running on the persistent resource.
Rebooting a persistent resource is a
long running operation,
during which the persistent resource can't be deleted. The operation contains a
progressMessage field that populates with an error status if one occurs. After
the operation indicates "done: true",
check the status
of the persistent resource. If the persistent resource is in the RUNNING
state, the reboot is successful and it's ready to run training jobs.
Limitations
The following are limitations for rebooting a persistent resource:
In some cases, it's possible to lose capacity of scarce resources when
rebooting a persistent resource. Full resource retention is not guaranteed.
Reboot is not available on Ray on Vertex AI.
Persistent resources containing autoscaled worker pools reboot with the
minimum replica count.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-26 UTC."],[],[],null,["You can reboot any persistent resource that's in the `RUNNING` or `ERROR` state. Rebooting a persistent resource lets you recover from errors that the persistent resource can't recover from on its own. You can also reboot a persistent resource to manually obtain more up-to-date clusters. This page shows you how to reboot a persistent resource by using the Google Cloud console and the REST API.\n\n\u003cbr /\u003e\n\nRequired roles\n\n\nTo get the permission that\nyou need to reboot a persistent resource,\n\nask your administrator to grant you the\n\n\n[Vertex AI Administrator](/iam/docs/roles-permissions/aiplatform#aiplatform.admin) (`roles/aiplatform.admin`)\nIAM role on your project.\n\n\nFor more information about granting roles, see [Manage access to projects, folders, and organizations](/iam/docs/granting-changing-revoking-access).\n\n\nThis predefined role contains the\n` aiplatform.persistentResources.update`\npermission,\nwhich is required to\nreboot a persistent resource.\n\n\nYou might also be able to get\nthis permission\nwith [custom roles](/iam/docs/creating-custom-roles) or\nother [predefined roles](/iam/docs/roles-overview#predefined).\n\nReboot a persistent resource\n\nSelect one of the following tabs for instructions on how to reboot a persistent\nresource. Make sure there's no training jobs running on the persistent resource. \n\nConsole\n\nTo reboot a persistent resource in the Google Cloud console, do the following:\n\n1. In the Google Cloud console, go to the **Persistent resources** page.\n\n [Go to Persistent resources](https://console.cloud.google.com/vertex-ai/training/persistent-resources)\n2. Next to the name of the persistent resource that you want to reboot, click\n the vertical ellipses (more_vert).\n\n3. Click **Reboot**.\n\n4. Click **Confirm**.\n\n\ngcloud\n\n\nBefore using any of the command data below,\nmake the following replacements:\n\n- \u003cvar class=\"edit\" scope=\"PROJECT_ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e: The Project ID of the persistent resource that you want to reboot.\n- \u003cvar class=\"edit\" scope=\"LOCATION\" translate=\"no\"\u003eLOCATION\u003c/var\u003e: The region of the persistent resource that you want to reboot.\n- \u003cvar class=\"edit\" scope=\"PERSISTENT_RESOURCE_ID\" translate=\"no\"\u003ePERSISTENT_RESOURCE_ID\u003c/var\u003e: The ID of the persistent resource that you want to reboot.\n\n\nExecute the\n\nfollowing\n\ncommand:\n\nLinux, macOS, or Cloud Shell **Note:** Ensure you have initialized the Google Cloud CLI with authentication and a project by running either [gcloud init](/sdk/gcloud/reference/init); or [gcloud auth login](/sdk/gcloud/reference/auth/login) and [gcloud config set project](/sdk/gcloud/reference/config/set). \n\n```bash\ngcloud ai persistent-resources reboot PERSISTENT_RESOURCE_ID \\\n --project=PROJECT_ID \\\n --region=LOCATION\n```\n\nWindows (PowerShell) **Note:** Ensure you have initialized the Google Cloud CLI with authentication and a project by running either [gcloud init](/sdk/gcloud/reference/init); or [gcloud auth login](/sdk/gcloud/reference/auth/login) and [gcloud config set project](/sdk/gcloud/reference/config/set). \n\n```bash\ngcloud ai persistent-resources reboot PERSISTENT_RESOURCE_ID `\n --project=PROJECT_ID `\n --region=LOCATION\n```\n\nWindows (cmd.exe) **Note:** Ensure you have initialized the Google Cloud CLI with authentication and a project by running either [gcloud init](/sdk/gcloud/reference/init); or [gcloud auth login](/sdk/gcloud/reference/auth/login) and [gcloud config set project](/sdk/gcloud/reference/config/set). \n\n```bash\ngcloud ai persistent-resources reboot PERSISTENT_RESOURCE_ID ^\n --project=PROJECT_ID ^\n --region=LOCATION\n```\n\nYou should receive a response similar to the following:\n\n```\nUsing endpoint [https://us-central1-aiplatform.googleapis.com/]\nRequest to reboot the PersistentResource [projects/sample-project/locations/us-central1/persistentResources/test-persistent-resource] has been sent.\n\nYou may view the status of your persistent resource with the command\n\n $ gcloud ai persistent-resources describe projects/sample-project/locations/us-central1/persistentResources/test-persistent-resource\n```\n\nREST\n\n\nBefore using any of the request data,\nmake the following replacements:\n\n- \u003cvar class=\"edit\" scope=\"PROJECT_ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e: The Project ID of the persistent resource that you want to reboot.\n- \u003cvar class=\"edit\" scope=\"LOCATION\" translate=\"no\"\u003eLOCATION\u003c/var\u003e: The region of the persistent resource that you want to reboot.\n- \u003cvar class=\"edit\" scope=\"PERSISTENT_RESOURCE_ID\" translate=\"no\"\u003ePERSISTENT_RESOURCE_ID\u003c/var\u003e: The ID of the persistent resource that you want to reboot.\n\n\nHTTP method and URL:\n\n```\nPOST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/persistentResources/PERSISTENT_RESOURCE_ID:reboot\n```\n\nTo send your request, expand one of these options:\n\ncurl (Linux, macOS, or Cloud Shell) **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](/sdk/gcloud/reference/init) or [`gcloud auth login`](/sdk/gcloud/reference/auth/login) , or by using [Cloud Shell](/shell/docs), which automatically logs you into the `gcloud` CLI . You can check the currently active account by running [`gcloud auth list`](/sdk/gcloud/reference/auth/list).\n\n\nExecute the following command:\n\n```\ncurl -X POST \\\n -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n -H \"Content-Type: application/json; charset=utf-8\" \\\n -d \"\" \\\n \"https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/persistentResources/PERSISTENT_RESOURCE_ID:reboot\"\n```\n\nPowerShell (Windows) **Note:** The following command assumes that you have logged in to the `gcloud` CLI with your user account by running [`gcloud init`](/sdk/gcloud/reference/init) or [`gcloud auth login`](/sdk/gcloud/reference/auth/login) . You can check the currently active account by running [`gcloud auth list`](/sdk/gcloud/reference/auth/list).\n\n\nExecute the following command:\n\n```\n$cred = gcloud auth print-access-token\n$headers = @{ \"Authorization\" = \"Bearer $cred\" }\n\nInvoke-WebRequest `\n -Method POST `\n -Headers $headers `\n -Uri \"https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/persistentResources/PERSISTENT_RESOURCE_ID:reboot\" | Select-Object -Expand Content\n```\n\nYou should receive a JSON response similar to the following:\n\n```\nresponse: \n {\n \"name\": \"projects/123456789012/locations/us-central1/persistentResources/test-persistent-resource/operations/1234567890123456789\",\n \"metadata\": {\n \"@type\": \"type.googleapis.com/google.cloud.aiplatform.v1.RebootPersistentResourceOperationMetadata\",\n \"genericMetadata\": {\n \"createTime\": \"2024-03-18T17:31:54.955004Z\",\n \"updateTime\": \"2024-03-18T17:31:55.204817Z\",\n \"state\": \"RUNNING\",\n \"worksOn\": [\n \"projects/123456789012/locations/us-central1/persistentResources/test-persistent-resource\"\n ]\n },\n \"progressMessage\": \"Waiting for persistent resource shut down.\"\n }\n }\n```\n\n\u003cbr /\u003e\n\nRebooting a persistent resource is a\n[long running operation](/vertex-ai/docs/general/long-running-operations),\nduring which the persistent resource can't be deleted. The operation contains a\n`progressMessage` field that populates with an error status if one occurs. After\nthe operation indicates `\"done: true\"`,\n[check the status](/vertex-ai/docs/training/persistent-resource-get#get_information_about_a_persistent_resource)\nof the persistent resource. If the persistent resource is in the `RUNNING`\nstate, the reboot is successful and it's ready to run training jobs.\n\nLimitations\n\nThe following are limitations for rebooting a persistent resource:\n\n- In some cases, it's possible to lose capacity of scarce resources when rebooting a persistent resource. Full resource retention is not guaranteed.\n- Reboot is not available on Ray on Vertex AI.\n- Persistent resources containing autoscaled worker pools reboot with the minimum replica count.\n\nWhat's next\n\n- [Learn about persistent resource](/vertex-ai/docs/training/persistent-resource-overview).\n- [Create and use a persistent resource](/vertex-ai/docs/training/persistent-resource-create).\n- [Run training jobs on a persistent resource](/vertex-ai/docs/training/persistent-resource-train).\n- [Get information about a persistent resource](/vertex-ai/docs/training/persistent-resource-get).\n- [Delete a persistent resource](/vertex-ai/docs/training/persistent-resource-delete)."]]
