Migrate from user-managed notebooks to Vertex AI Workbench instances

This page describes how to migrate from a user-managed notebooks instance to a Vertex AI Workbench instance. You can migrate by using the Vertex AI Workbench migration tool or migrate your instance's data and files manually.

Overview of the migration tool

Vertex AI Workbench provides a migration tool for migrating from a user-managed notebooks instance to a Vertex AI Workbench instance.

The migration tool creates a Vertex AI Workbench instance with a configuration similar to the user-managed notebooks instance that you want to migrate. For example, the migration tool creates an instance that has the same or similar machine type, network configuration, idle shutdown settings, and other specifications. Then, the files on your user-managed notebooks instance's data disk are copied to the Vertex AI Workbench instance.

Vertex AI Workbench doesn't delete or change your user-managed notebooks instance, so after migration you can continue to use it. If you don't need the user-managed notebooks instance anymore, delete it to avoid further charges for that instance.

Billing

If your user-managed notebooks instance uses Extreme Persistent Disks, then your migration generates charges for I/O operations. See "Extreme provisioned IOPS" in the Persistent Disk and Hyperdisk pricing section of Disk pricing.

After migration, the user-managed notebooks instance still exists and generates charges as before. If you don't need the user-managed notebooks instance anymore, delete it to avoid further charges for that instance.

Default migration tool behaviors

The Vertex AI Workbench migration tool attempts to migrate your user-managed notebooks instance to a Vertex AI Workbench instance with matching specifications. When a specification in your user-managed notebooks instance isn't available in Vertex AI Workbench instances, Vertex AI Workbench uses a default specification when possible. When the migration tool can't migrate a specification of your user-managed notebooks instance, it doesn't migrate the instance.

The following table lists some of the key default migration behaviors for the migration tool.

Specifying the post-startup script

User-managed notebooks instances that use a post-startup script must be migrated to an instance with the PostStartupScriptOption option specified. Use this option to indicate whether you want to skip or rerun the post-startup script in your new Vertex AI Workbench instance.

Specifying the PostStartupScriptOption option isn't supported in the Google Cloud console. To specify the PostStartupScriptOption option when you migrate your user-managed notebooks instance, you must use the Google Cloud CLI or REST API.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Notebooks API.

   Enable the API

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the Notebooks API.

   Enable the API

8. If you haven't already, create a user-managed notebooks instance.

Required roles

To ensure that your user account has the necessary permissions to migrate a user-managed notebooks instance to a Vertex AI Workbench instance, ask your administrator to grant your user account the Notebooks Runner (roles/notebooks.runner) IAM role on the project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to migrate a user-managed notebooks instance to a Vertex AI Workbench instance. To see the exact permissions that are required, expand the Required permissions section:

Your administrator might also be able to give your user account these permissions with custom roles or other predefined roles.

Pre-migration check

Before you migrate, check your user-managed notebooks instance's migration eligibility by listing your instances and checking the output for any migration warnings or errors.

List your instances

To list your user-managed notebooks instances that aren't migrated yet, use the projects.locations.instances.list method with the filter migrated:false. You can list them by using the gcloud CLI or REST API:

gcloud notebooks instances list --project=PROJECT_ID \
    --location=LOCATION --filter=migrated:false --format=default

gcloud notebooks instances list --project=PROJECT_ID `
    --location=LOCATION --filter=migrated:false --format=default

gcloud notebooks instances list --project=PROJECT_ID ^
    --location=LOCATION --filter=migrated:false --format=default

GET https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?filter=migrated:false

curl -X GET \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?filter=migrated:false"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?filter=migrated:false" | Select-Object -Expand Content

Check the output for warnings or errors

If migration warnings or errors are detected, the output of the projects.locations.instances.list method includes this information.

Warnings appear when specific components in your user-managed notebooks instance's configuration won't migrate to the same specification in a Vertex AI Workbench instance. For example, if your user-managed notebooks instance uses an unsupported accelerator, a warning appears in the output. In this case, the instance is migrated without any accelerators. You can attach accelerators after migration. Review the warnings in the output, consider the migration tool's default behaviors, and assess whether the migration tool is acceptable for your migration.

One or more errors in the output means that you can't migrate the user-managed notebooks instance by using the migration tool. You must migrate the instance manually.

For more information about migration warnings and errors, see warnings and errors in the InstanceMigrationEligibility documentation.

Migrate by using the migration tool

You can migrate your user-managed notebooks instance by using the Google Cloud console, the gcloud CLI, or REST API.

gcloud notebooks instances migrate RUNTIME_ID \
    --project=PROJECT_ID \
    --location=LOCATION \
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

gcloud notebooks instances migrate RUNTIME_ID `
    --project=PROJECT_ID `
    --location=LOCATION `
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

gcloud notebooks instances migrate RUNTIME_ID ^
    --project=PROJECT_ID ^
    --location=LOCATION ^
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

POST https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/INSTANCE_ID:migrate

{
  "postStartupScriptOption": (POST_STARTUP_SCRIPT_OPTION)
}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/INSTANCE_ID:migrate"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/INSTANCE_ID:migrate" | Select-Object -Expand Content

Migrate manually

To migrate your instance to a Vertex AI Workbench instance manually, consider using the following methods:

• Use Cloud Storage and the terminal: Copy your data and files to Cloud Storage and then to another instance by using the terminal.

• Use GitHub: Copy your data and files to a GitHub repository by using the Git extension for JupyterLab.

This guide describes how to migrate data and files by using Cloud Storage and the terminal.

Requirements

You must have terminal access to your user-managed notebooks instance. Terminal access is manually set when you create an instance. The terminal access setting cannot be changed after the instance is created.

Migrate manually by using Cloud Storage and the terminal

To migrate data and files to a new Vertex AI Workbench instance by using Cloud Storage and the terminal, do the following.

1. Create a Cloud Storage bucket in the same project where your user-managed notebooks instance is located.

2. In that same project, Create a Vertex AI Workbench instance to migrate your data to. When you create this instance:

   • Enable terminal access.
   • Specify the machine type, network, and other characteristics to match what you need.

3. In your user-managed notebooks instance's JupyterLab interface, select File > New > Terminal to open a terminal window.

4. Use the gcloud CLI to copy your user data to a Cloud Storage bucket. The following example command copies all of the files from your instance's /home/jupyter/ directory to a directory in a Cloud Storage bucket.

   gcloud storage cp /home/jupyter/* gs://BUCKET_NAMEPATH --recursive

   Replace the following:

   • BUCKET_NAME: The name of your Cloud Storage bucket
   • PATH: The path to the directory where you want to copy your files, for example: /copy/jupyter/

5. In your new Vertex AI Workbench instance's JupyterLab interface, select File > New > Terminal to open a terminal window.

6. Use the gcloud CLI to copy your data to the new instance. The following example command copies all of the files from a Cloud Storage directory to the your new instance's /home/jupyter/ directory.

   gcloud storage cp gs://BUCKET_NAMEPATH* /home/jupyter/

Confirm the migration

After the migration, the original user-managed notebooks instance continues to work as before. Confirm that your migration was a success before you delete the original instance.

Delete the user-managed notebooks instance

If you don't need the user-managed notebooks instance that you migrated from, delete it to avoid further charges for that instance.

1. In the Google Cloud console, go to the User-managed notebooks page.

   Go to User-managed notebooks

2. Select the instance that you want to delete.

3. Click delete Delete. (Depending on the size of your window, the Delete button might be in the more_vert options menu.)

4. To confirm, click Delete.

Troubleshoot

To find methods for diagnosing and resolving migration issues, see Troubleshooting Vertex AI Workbench.

What's next

• Learn more about Vertex AI Workbench instances.