Create GPU VMs in bulk

Stay organized with collections Save and categorize content based on your preferences.

gcloud

To create a group of VMs, use the gcloud compute instances bulk create command. For more information about the parameters and how to use this command, see Create VMs in bulk.

The following optional flags are shown in the example command:

• The --provisioning-model=SPOT is an optional flag that configures your VMs as Spot VMs. If your workload is fault-tolerant and can withstand possible VM preemption, consider using Spot VMs to reduce the cost of your VMs and the attached GPUs. For more information, see GPUs on Spot VMs. For Spot VMs, the automatic restart and host maintenance options flags are disabled.

• The --accelerator flag to specify a virtual workstation. NVIDIA RTX Virtual Workstations (vWS) are supported for only G2 VMs.

Example

This example creates two VMs that have attached GPUs by using the following specifications:

• VM names: my-test-vm-1, my-test-vm-2
• Each VM has two GPUs attached, specified by using the appropriate accelerator-optimized machine type

gcloud compute instances bulk create \
    --name-pattern="my-test-vm-#" \
    --region=REGION \
    --count=2 \
    --machine-type=MACHINE_TYPE \
    --boot-disk-size=200 \
    --image=IMAGE \
    --image-project=IMAGE_PROJECT \
    --on-host-maintenance=TERMINATE \
    [--provisioning-model=SPOT] \
    [--accelerator=type=nvidia-l4-vws,count=VWS_ACCELERATOR_COUNT]

Replace the following:

• REGION: the region for the VMs. This region must support your selected GPU model.

• MACHINE_TYPE: the machine type that you selected. Choose from one of the following:

  • An A3 machine type.
  • An A2 machine type.
  • A G2 machine type. G2 machine types also support custom memory. Memory must be a multiple of 1024 MB and within the supported memory range. For example, to create a VM with 4 vCPUs and 19 GB of memory specify --machine-type=g2-custom-4-19456.

• IMAGE: an operating system image that supports GPUs.

  If you want to use the latest image in an image family, replace the --image flag with the --image-family flag and set its value to an image family that supports GPUs. For example: --image-family=rocky-linux-8-optimized-gcp.

  You can also specify a custom image or Deep Learning VM Images.

• IMAGE_PROJECT: the Compute Engine image project that the OS image belongs to. If using a custom image or Deep Learning VM Images, specify the project that those images belong to.

• VWS_ACCELERATOR_COUNT: the number of virtual GPUs that you need.

If successful, the output is similar to the following:

NAME          ZONE
my-test-vm-1  us-central1-b
my-test-vm-2  us-central1-b
Bulk create request finished with status message: [VM instances created: 2, failed: 0.]

REST

Use the instances.bulkInsert method with the required parameters to create multiple VMs in a zone. For more information about the parameters and how to use this command, see Create VMs in bulk.

Example

This example creates two VMs that have attached GPUs by using the following specifications:

• VM names: my-test-vm-1, my-test-vm-2

• Each VM has two GPUs attached, specified by using the appropriate accelerator-optimized machine type

  POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instances/bulkInsert
  {
  "namePattern":"my-test-vm-#",
  "count":"2",
  "instanceProperties": {
    "machineType":MACHINE_TYPE,
    "disks":[
      {
        "type":"PERSISTENT",
        "initializeParams":{
          "diskSizeGb":"200",
          "sourceImage":SOURCE_IMAGE_URI
        },
        "boot":true
      }
    ],
    "name": "default",
    "networkInterfaces":
    [
      {
        "network": "projects/PROJECT_ID/global/networks/default"
      }
    ],
    "scheduling":{
      "onHostMaintenance":"TERMINATE",
      ["automaticRestart":true]
    }
  }
  }
  

Replace the following:

• PROJECT_ID: your project ID
• REGION: the region for the VMs. This region must support your selected GPU model.

• MACHINE_TYPE: the machine type that you selected. Choose from one of the following:

  • An A2 machine type.
  • A G2 machine type. G2 machine types also support custom memory. Memory must be a multiple of 1024 MB and within the supported memory range. For example, to create a VM with 4 vCPUs and 19 GB of memory specify --machine-type=g2-custom-4-19456.

• SOURCE_IMAGE_URI: the URI for the specific image or image family that you want to use.

  For example:

  • Specific image: "sourceImage": "projects/rocky-linux-cloud/global/images/rocky-linux-8-optimized-gcp-v20220719"
  • Image family: "sourceImage": "projects/rocky-linux-cloud/global/images/family/rocky-linux-8-optimized-gcp".

  When you specify an image family, Compute Engine creates a VM from the most recent, non-deprecated OS image in that family. For more information about when to use image families, see Image family best practices.

Additional settings:

• If your workload is fault-tolerant and can withstand possible VM preemption, consider using Spot VMs to reduce the cost of your VMs and the attached GPUs. For more information, see GPUs on Spot VMs. To use a Spot VM, add the "provisioningModel": "SPOT option to your request. For Spot VMs, the automatic restart and host maintenance options flags are disabled.

  "scheduling":
    {
      "provisioningModel": "SPOT"
    }
  

• For G2 VMs, NVIDIA RTX Virtual Workstations (vWS) are supported. To specify a virtual workstation, add the guestAccelerators option to your request. Replace VWS_ACCELERATOR_COUNT with the number of virtual GPUs that you need.

  "guestAccelerators":
   [
     {
       "acceleratorCount": VWS_ACCELERATOR_COUNT,
       "acceleratorType": "projects/PROJECT_ID/zones/ZONEacceleratorTypes/nvidia-l4-vws"
     }
   ]
  

gcloud

To create a group of VMs, use the gcloud compute instances bulk create command. For more information about the parameters and how to use this command, see Create VMs in bulk.

Example

The following example creates two VMs with attached GPUs using the following specifications:

• VM names: my-test-vm-1, my-test-vm-2
• VMs created in any zone in us-central1 that supports GPUs
• Each VM has two T4 GPUs attached, specified by using the accelerator type and accelerator count flags
• Each VM has GPU drivers installed
• Each VM uses the Deep Learning VM image pytorch-latest-gpu-v20211028-debian-10

gcloud compute instances bulk create \
    --name-pattern="my-test-vm-#" \
    --count=2 \
    --region=us-central1 \
    --machine-type=n1-standard-2 \
    --accelerator type=nvidia-tesla-t4,count=2 \
    --boot-disk-size=200 \
    --metadata="install-nvidia-driver=True" \
    --scopes="https://www.googleapis.com/auth/cloud-platform" \
    --image=pytorch-latest-gpu-v20211028-debian-10 \
    --image-project=deeplearning-platform-release \
    --on-host-maintenance=TERMINATE --restart-on-failure

If successful, the output is similar to the following:

NAME          ZONE
my-test-vm-1  us-central1-b
my-test-vm-2  us-central1-b
Bulk create request finished with status message: [VM instances created: 2, failed: 0.]

REST

Use the instances.bulkInsert method with the required parameters to create multiple VMs in a zone. For more information about the parameters and how to use this command, see Create VMs in bulk.

Example

The following example creates two VMs with attached GPUs using the following specifications:

• VM names: my-test-vm-1, my-test-vm-2
• VMs created in any zone in us-central1 that supports GPUs
• Each VM has two T4 GPUs attached, specified by using the accelerator type and accelerator count flags
• Each VM has GPU drivers installed
• Each VM uses the Deep Learning VM image pytorch-latest-gpu-v20211028-debian-10

Replace PROJECT_ID with your project ID.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/region/us-central1/instances/bulkInsert

{
    "namePattern":"my-test-vm-#",
    "count":"2",
    "instanceProperties": {
      "machineType":"n1-standard-2",
      "disks":[
        {
          "type":"PERSISTENT",
          "initializeParams":{
            "diskSizeGb":"200",
            "sourceImage":"projects/deeplearning-platform-release/global/images/pytorch-latest-gpu-v20211028-debian-10"
          },
          "boot":true
        }
      ],
      "name": "default",
      "networkInterfaces":
      [
        {
          "network": "projects/PROJECT_ID/global/networks/default"
        }
      ],
      "guestAccelerators":
      [
        {
          "acceleratorCount": 2,
          "acceleratorType": "nvidia-tesla-t4"
        }
      ],
      "scheduling":{
        "onHostMaintenance":"TERMINATE",
        "automaticRestart":true
      },
      "metadata":{
        "items":[
          {
            "key":"install-nvidia-driver",
            "value":"True"
          }
        ]
      }
  }
 }