Add or remove GPUs

Console

You can modify the number of GPUs for your VM by stopping the VM and editing the VM configuration.

1. Verify that all of your critical applications are stopped on the VM.

2. In the Google Cloud console, go to the VM instances page to see your list of VMs.

   Go to VM instances

3. Click the name of the VM that you want to modify the number of GPUs for. The Details page opens.

4. Complete the following steps from the Details page.

   1. If the VM is running, on the toolbar, click stop Stop. Then, wait for the VM to stop.

   2. On the toolbar, click edit Edit.

   3. In the Machine configuration section, select the GPUs machine family, and then do the following:

      1. In the Number of GPUs list, increase or decrease the GPU count.

   4. To apply your changes, click Save.

   5. To restart the VM, click Start/Resume.

REST

You can modify the number of GPUs on your VM by stopping the VM and changing the machine type. Each accelerator-optimized machine type has a specific number of GPUs attached. If you change the machine type, this adjusts the number of GPUs that are attached to the VM.

1. Verify that all of your critical applications are stopped on the VM, and then create a POST command to stop the VM so it can move to a host system where GPUs are available.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/stop
   

2. After the VM stops, create a POST request to modify the machine type.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/setMachineType
   
   {
      machineType: "zones/ZONE/machineTypes/MACHINE_TYPE"
   }
   

3. Start the VM.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/start
   

Replace the following:

• PROJECT_ID: your project ID.
• VM_NAME: the name of the VM that you want to add GPUs to.
• ZONE: the zone where the VM is located. This zone must support GPUs.

• MACHINE_TYPE: the machine type that you want to use. It must be one of the following:

  • If your VM uses an A2 standard machine, select another A2 machine type.
  • If your VM uses a G2 machine type, select another G2 machine type. G2 machine types also support custom memory. Memory must be a multiple of 1024 MB and within the supported memory range. For example, to create a VM with 4 vCPUs and 19 GB of memory specify --machine-type=g2-custom-4-19456.

A2 VMs

• You don't receive sustained use discounts and flexible committed use discounts for VMs that use A2 Standard machine types.
• You can only use A2 Standard machine types in certain regions and zones.
• You can't use regional persistent disks on VMs that use A2 Standard machine types.
• The A2 Standard machine type is only available on the Cascade Lake platform.
• If your VM uses an A2 Standard machine type, you can only switch from one A2 Standard machine type to another A2 Standard machine type. You can't change to any other machine type. For more information, see Modify accelerator-optmized VMs.
• You can't use the Windows operating system with A2 Standard machine types. When using Windows operating systems, choose a different A2 Standard machine type.
• You can't do a quick format of the attached Local SSDs on Windows VMs that use A2 Standard machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.
• A2 Standard machine types don't support sole-tenancy.

G2 VMs

• You don't receive sustained use discounts and flexible committed use discounts for VMs that use G2 machine types.
• You can only use G2 machine types in certain regions and zones.
• You can't use regional persistent disks on VMs that use G2 machine types.
• The G2 machine type is only available on the Cascade Lake platform.
• Standard persistent disks (pd-standard) are not supported on VMs that use G2 standard machine types. For supported disk types, see Supported disk types for G2.
• You can't create Multi-Instance GPUs on G2 machine types.
• If you need to change the machine type of a G2 VM, review Modify accelerator-optmized VMs.
• You can't use Deep Learning VM Images as boot disks for your VMs that use G2 machine types.
• The current default driver for Container-Optimized OS doesn't support L4 GPUs running on G2 machine types. Container-Optimized OS also only support a select set of drivers. If you want to use Container-Optimized OS on G2 machine types, review the following notes:
  • Use a Container-Optimized OS version that supports the minimum recommended NVIDIA driver version 525.60.13 or later. For more information, review the Container-Optimized OS release notes.
  • When you install the driver, specify the latest available version that works for the L4 GPUs. For example, sudo cos-extensions install gpu -- -version=525.60.13.
• You must use the Google Cloud CLI or REST to create G2 VMs for the following scenarios:
  • You want to specify custom memory values.
  • You want to customize the number of visible CPU cores.

Console

To add GPUs or modify the GPU type, complete the following steps.

1. Verify that all of your critical applications are stopped on the VM.

2. In the Google Cloud console, go to the VM instances page to see your list of VMs.

   Go to VM instances

3. Click the name of the VM that you want to update. The Details page opens.

4. Complete the following steps from the Details page.

   1. If the VM is running, on the toolbar, click stop Stop. Then, wait for the VM to stop.

   2. On the toolbar, click edit Edit.

   3. In the Machine configuration section, select the GPUs machine family, and then do the following:

      1. In the GPU type list, select or switch to any of the GPU types supported on N1 VMs.

      2. In the Number of GPUs list, select the number of GPUs.

      3. If your GPU model supports NVIDIA RTX Virtual Workstations (vWS) for graphics workloads, and you plan on running graphics-intensive workloads on this VM, select Enable Virtual Workstation (NVIDIA GRID).

   4. If your VM didn't have GPUs attached before, complete the following:

      1. If the VM has a shared-core machine type, you must change the machine type. In the Machine type list, select one of the preset N1 machine types. Alternatively, you can also specify custom machine type settings.

      2. In the Management section, complete the following:

         1. In the On host maintenance list, select Terminate VM instance. VMs with attached GPUs can't live migrate. See Handle GPU host events.

         2. In the Automatic restart list, select On.

   5. To apply your changes, click Save.

   6. To restart the VM, click Start/Resume.

REST

You can add or modify GPUs on your VM by stopping the VM and changing your VM's configuration through the API.

1. Verify that all of your critical applications are stopped on the VM and then create a POST command to stop the VM so it can move to a host system where GPUs are available.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/stop
   

2. If your VM didn't have GPUs attached before, complete the following steps:

   1. Identify the GPU type that you want to add to your VM. You can submit a GET request to list the GPU types that are available to your project in a specific zone.

      GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/acceleratorTypes
      

   2. If the VM has a shared-core machine type, you must change the machine type to have one or more vCPUs. You cannot add accelerators to VMs with shared-core machine types.

   3. Create a POST command to set the scheduling options for the VM.

      POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/setScheduling
      
      {
      "onHostMaintenance": "TERMINATE",
      "automaticRestart": true
      }
      

3. Create a POST request to add or modify the GPUs that are attached to your VM.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/setMachineResources
   
       {
        "guestAccelerators": [
         {
           "acceleratorCount": ACCELERATOR_COUNT,
           "acceleratorType": "https://www.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/acceleratorTypes/ACCELERATOR_TYPE"
         }
        ]
       }
   

4. Start the VM.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/start
   

   Replace the following:

   • PROJECT_ID: your project ID.
   • VM_NAME: the name of the VM that you want to add GPUs to.
   • ZONE: the zone where the VM is located.
   • ACCELERATOR_COUNT: the number of GPUs that you want attached to your VM. For a list of GPU limits based on the machine type of your VM, see GPUs on Compute Engine.

   • ACCELERATOR_TYPE: the GPU model that you want to attach or switch to. If you plan on running graphics-intensive workloads on this VM, use one of the virtual workstation models.

     Choose one of the following values:

     • NVIDIA GPUs:

       • NVIDIA T4: nvidia-tesla-t4
       • NVIDIA P4: nvidia-tesla-p4
       • NVIDIA P100: nvidia-tesla-p100
       • NVIDIA V100: nvidia-tesla-v100

     • NVIDIA RTX Virtual Workstation (vWS) (formerly known as NVIDIA GRID):

       • NVIDIA T4 Virtual Workstation: nvidia-tesla-t4-vws
       • NVIDIA P4 Virtual Workstation: nvidia-tesla-p4-vws
       • NVIDIA P100 Virtual Workstation: nvidia-tesla-p100-vws

       For these virtual workstations, an NVIDIA RTX Virtual Workstation (vWS) license is automatically added to your VM.