Manage TPU Spot VMs

Spot VMs make unused capacity available at highly-discounted rates. Spot VMs can be preempted (shut down) at any time, but unlike preemptible TPUs, there is no limit on runtime duration. You can't restart TPU Spot VMs, and you must recreate them after preemption. For more information about Spot VMs in general, see the Compute Engine documentation about Spot VMs.

Create TPU Spot VMs

You can create TPU Spot VMs using queued resources, which adds your create request to a queue and lets you receive capacity once it becomes available. Creating TPUs as queued resources is a best practice. For more information, see Queued resources user guide.

The following example shows how to create TPU Spot VMs using queued resources:

gcloud

Add the --spot flag to the gcloud compute tpus queued-resources command:

gcloud compute tpus queued-resources create QUEUED_RESOURCE_ID \
    --node-id=NODE_ID \
    --zone=europe-west4-a \
    --accelerator-type=v3-8 \
    --runtime-version=tpu-vm-tf-2.17.1-pod-pjrt \
    --spot

curl

Add spot: {} to the request body:

curl -X POST https://tpu.googleapis.com/v2alpha1/projects/PROJECT_ID/locations/europe-west4-a/queuedResources?queued_resource_id=QUEUED_RESOURCE_ID \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    -d '{
        tpu: {
        node_spec: {
            parent: "projects/PROJECT_ID/locations/europe-west4-a",
            node_id: "NODE_ID",
            node: {
            accelerator_type: "v3-8",
            runtime_version: "tpu-vm-tf-2.17.1-pod-pjrt"
            }
        }
        },
        spot: {}
    }'

Console

  1. In the Google Cloud console, go to the TPUs page:

    Go to TPUs

  2. Click Create TPU.

  3. In the Name field, enter a name for your TPU.

  4. In the Zone box, select the zone where you want to create the TPU.

  5. In the TPU type box, select an accelerator type. The accelerator type specifies the version and size of the Cloud TPU you want to create. For more information about supported accelerator types for each TPU version, see TPU versions.

  6. In the TPU software version box, select a software version. When creating a Cloud TPU VM, the TPU software version specifies the version of the TPU runtime to install. For more information, see TPU software versions.

  7. Click the Enable queueing toggle.

  8. In the Queued resource name field, enter a name for your queued resource request.

  9. Expand the Management section.

  10. Select the Make this a TPU Spot VM checkbox.

  11. Click Create.

You can also create TPU Spot VMs without using queued resources:

gcloud

Add the --spot flag to the gcloud compute tpus tpu-vm command:

gcloud compute tpus tpu-vm create TPU_NAME \
    --zone=europe-west4-a \
    --accelerator-type=v3-8 \
    --version=tpu-vm-tf-2.17.1-pod-pjrt \
    --spot

curl

Add scheduling_config: {spot: true} to the request body:

curl -X POST https://tpu.googleapis.com/v2/projects/PROJECT_ID/locations/europe-west4-a/nodes?node_id=TPU_NAME \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    -d '{
        accelerator_type: "v3-8",
        runtime_version: "tpu-vm-tf-2.17.1-pod-pjrt",
        network_config: {enable_external_ips: true},
        scheduling_config: {spot: true}
    }'

Console

  1. In the Google Cloud console, go to the TPUs page:

    Go to TPUs

  2. Click Create TPU.

  3. In the Name field, enter a name for your TPU.

  4. In the Zone box, select the zone where you want to create the TPU.

  5. In the TPU type box, select an accelerator type. The accelerator type specifies the version and size of the Cloud TPU you want to create. For more information about supported accelerator types for each TPU version, see TPU versions.

  6. In the TPU software version box, select a software version. When creating a Cloud TPU VM, the TPU software version specifies the version of the TPU runtime to install. For more information, see TPU software versions.

  7. Expand the Management section.

  8. Select the Make this a TPU Spot VM checkbox.

  9. Click Create.

Check if a TPU VM is a Spot VM

gcloud

To confirm if your TPU VM is a Spot VM, use the describe command:

gcloud compute tpus tpu-vm describe TPU_NAME --zone=europe-west4-a

If the TPU VM is a Spot VM, then the output will include the spot field set to true, similar to the following:

...
schedulingConfig:
spot: true
...

Console

  1. In the Google Cloud console, go to the TPUs page:

    Go to TPUs

  2. Click the name of your Cloud TPU.

    If the TPU VM is a Spot VM, then the Tier will be Spot VM.

Pricing and quota

Pricing for TPU Spot VMs is significantly lower than for on-demand and reserved TPUs. For more information about pricing, see Cloud TPU pricing.

You need preemptible quota to use TPU Spot VMs. For more information, see Quotas.