Frequently Asked Questions - Cloud TPU
This document contains a list of frequently asked questions about Cloud TPUs. It is broken up into sections:
- Framework independent FAQs - questions about using Cloud TPUs regardless of what ML framework you are using.
- JAX FAQS - questions about using Cloud TPUs with JAX.
- PyTorch FAQs - questions about using Cloud TPUs with PyTorch.
Framework independent FAQs
How do I check which process is using the TPU on a Cloud TPU VM?
Run tpu-info
on the Cloud TPU VM to print the process ID and
other information about the process using the TPU. See supported metrics for the metrics
and their corresponding definitions.
tpu-info
The output from tpu-info
is similar to the following:
TPU Chips
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ Chip ┃ Type ┃ Devices ┃ PID ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ /dev/accel0 │ TPU v4 chip │ 1 │ 130007 │
│ /dev/accel1 │ TPU v4 chip │ 1 │ 130007 │
│ /dev/accel2 │ TPU v4 chip │ 1 │ 130007 │
│ /dev/accel3 │ TPU v4 chip │ 1 │ 130007 │
└─────────────┴─────────────┴─────────┴────────┘
TPU Runtime Utilization
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Device ┃ Memory usage ┃ Duty cycle ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ 0 │ 0.00 GiB / 31.75 GiB │ 0.00% │
│ 1 │ 0.00 GiB / 31.75 GiB │ 0.00% │
│ 2 │ 0.00 GiB / 31.75 GiB │ 0.00% │
│ 3 │ 0.00 GiB / 31.75 GiB │ 0.00% │
└────────┴──────────────────────┴────────────┘
TensorCore Utilization
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Chip ID ┃ TensorCore Utilization ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 0 │ 0.00% │
│ 1 │ 0.00% │
│ 3 │ 0.00% │
│ 2 │ 0.00% |
└─────────┴────────────────────────┘
Buffer Transfer Latency
┏━━━━━━━━━━━━━┳━━━━━┳━━━━━┳━━━━━┳━━━━━━┓
┃ Buffer Size ┃ P50 ┃ P90 ┃ P95 ┃ P999 ┃
┡━━━━━━━━━━━━━╇━━━━━╇━━━━━╇━━━━━╇━━━━━━┩
│ 8MB+ | 0us │ 0us │ 0us │ 0us |
└─────────────┴─────┴─────┴─────┴──────┘
How do I add a persistent disk volume to a Cloud TPU VM?
For more information, see Add a persistent disk to a TPU VM.
What storage options are supported or recommended for training with TPU VM?
For more information, see Cloud TPU storage options.
JAX FAQs
How do I know if the TPU is being used by my program?
There are a few ways to double check JAX is using the TPU:
Use the
jax.devices()
function. For example:assert jax.devices()[0].platform == 'tpu'
Profile your program and verify the profile contains TPU operations. For more information, see Profiling JAX programs
For more information, see JAX FAQ
Pytorch FAQs
How do I know if the TPU is being used by my program?
You can run following python commands:
>>> import torch_xla.core.xla_model as xm
>>> xm.get_xla_supported_devices(devkind="TPU")
And verify if you can see any TPU devices.