Create and run a job that uses custom boot disks

This page describes how to customize the persistent disks used to boot each virtual machine (VM) instance that a job runs on.

Specifically, you can customize the size, type, and/or the VM operating system (OS) image for a boot disk. If you only want to customize the VM OS image, see Specify the VM OS image for a job instead.

To learn more about boot disks and when to configure boot disks, see VM OS environment overview.

Before you begin

If you haven't used Batch before, review Get started with Batch and enable Batch by completing the prerequisites for projects and users.
To get the permissions that you need to create a job, ask your administrator to grant you the following IAM roles:
- Batch Job Editor (roles/batch.jobsEditor) on the project
- Service Account User (roles/iam.serviceAccountUser) on the job's service account, which by default is the default Compute Engine service account
For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Create and run a job that uses a custom boot disk

Specify a custom boot disk when you are creating a job by selecting one of the following methods:

Use a Compute Engine instance template that specifies a pre-existing custom boot disk. If you want to use a VM instance template while creating this job or use a pre-existing custom boot disk, complete the following steps:
1. Create or identify a custom boot disk.
  
  For boot disk requirements for Batch, see this document and see VM OS environment overview. For instructions on how to create a boot disk, see Create a customized boot disk in the Compute Engine documentation.
2. Create or identify a VM instance template that includes this custom boot disk.
  
  For instructions on how to create a VM instance template, see Create instance templates in the Compute Engine documentation.
3. Create and run a job that includes this VM instance template.
  
  For instructions on how to create and run the job, see Define job resources using a VM instance template.
Use the boot disk field to specify a new custom boot disk. As the following instructions explain, you can define a new custom boot disk by using the boot disk (bootDisk) field when you create and run a job using the gcloud CLI or Batch API.

Specifically, you can customize the boot disk by configuring the VM OS image (image), persistent disk type (type), and/or size (sizeGb) subfields. Batch sets a default value for any of these subfields that you choose to omit.

gcloud

Create a JSON file that specifies your job's configuration details. To specify a new custom boot disk for the job's VMs, include the bootDisk field and subfields.

For example, to create a basic script job that uses a new custom boot disk, create a JSON file with the following contents:
```
{
    "taskGroups": [
        {
            "taskSpec": {
                "runnables": [
                    {
                        "script": {
                            "text": "echo Hello world from task ${BATCH_TASK_INDEX}."
                        }
                    }
                ]
            },
            "taskCount": 3,
            "parallelism": 1
        }
    ],
    "allocationPolicy": {
      "instances": [
        {
          "policy": {
            "bootDisk": {
              "image": "VM_OS_IMAGE_URI",
              "type": "BOOT_DISK_TYPE",
              "sizeGb": BOOT_DISK_SIZE
            }
          }
        }
      ]
    },
    "logsPolicy": {
    "destination": "CLOUD_LOGGING"
  }
}
```
Replace the following:
- VM_OS_IMAGE_URI: a relative resource name of the VM OS image. Use one of the following options:
  - Specify a Batch OS prefix. To use the latest image of a specific Batch OS, use the following format:
```
BATCH_OS_PREFIX
```
    Replace BATCH_OS_PREFIX with one of the Batch VM OS image prefixes—for example, batch-debian is the prefix for the Batch Debian OS.
  - Specify an image family. To use the latest image from a specific image family, use the following format:
```
projects/IMAGE_PROJECT_ID/global/images/family/IMAGE_FAMILY
```
    Replace the following:
    - IMAGE_PROJECT_ID: the project ID of the project that contains the image. For example, for all Batch images, specify batch-custom-image.
    - IMAGE_FAMILY: the image family, which includes one or more specific images and represents the type and major version of the OS. For example, to see all the VM OS image families from Batch, view a list of VM OS images.
  - Specify an image version. To use a specific version of a VM OS image, use the following format:
```
projects/IMAGE_PROJECT_ID/global/images/IMAGE_NAME
```
    Replace the following:
    - IMAGE_PROJECT_ID: the project ID of the project that contains the image. For example, for all Batch images, specify batch-custom-image.
    - IMAGE_NAME: the image name, which represents a specific version of a VM OS image. For example, to see all the VM OS image versions from Batch, view a list of VM OS images.
- BOOT_DISK_TYPE: the disk type of the boot disk, either pd-standard, pd-balanced, pd-ssd, or pd-extreme. The default disk type for boot disks is pd-balanced.
- BOOT_DISK_SIZE: the size of the new persistent boot disk in GB. This value must comply with all the boot disk size requirements.
To create the job, run the following gcloud batch jobs submit command:
```
gcloud batch jobs submit JOB_NAME \
    --location LOCATION \
    --config JSON_CONFIGURATION_FILE
```
Replace the following:
- JOB_NAME: the name for this job.
- LOCATION: the location for this job.
- JSON_CONFIGURATION_FILE: the path to the JSON file with the job's configuration details.

API

To create a job using the Batch API, use the jobs.create method and specify your job's configuration details. To specify the VM OS image for the job, include the bootDisk field and subfields. For example, to create a basic script job that uses a new custom boot disk, make the following POST request:

POST https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/jobs?job_id=JOB_NAME

{
    "taskGroups": [
        {
            "taskSpec": {
                "runnables": [
                    {
                        "script": {
                            "text": "echo Hello world from task ${BATCH_TASK_INDEX}."
                        }
                    }
                ]
            },
            "taskCount": 3,
            "parallelism": 1
        }
    ],
    "allocationPolicy": {
      "instances": [
        {
          "policy": {
            "bootDisk": {
              "image": "VM_OS_IMAGE_URI",
              "type": "BOOT_DISK_TYPE",
              "sizeGb": BOOT_DISK_SIZE
            }
          }
        }
      ]
    },
    "logsPolicy": {
    "destination": "CLOUD_LOGGING"
  }
}

Replace the following:

VM_OS_IMAGE_URI: a relative resource name of the VM OS image. Use one of the following options:
- Specify a Batch OS prefix. To use the latest image of a specific Batch OS, use the following format:
```
BATCH_OS_PREFIX
```
  Replace BATCH_OS_PREFIX with one of the Batch VM OS image prefixes—for example, batch-debian is the prefix for the Batch Debian OS.
- Specify an image family. To use the latest image from a specific image family, use the following format:
```
projects/IMAGE_PROJECT_ID/global/images/family/IMAGE_FAMILY
```
  Replace the following:
  - IMAGE_PROJECT_ID: the project ID of the project that contains the image. For example, for all Batch images, specify batch-custom-image.
  - IMAGE_FAMILY: the image family, which includes one or more specific images and represents the type and major version of the OS. For example, to see all the VM OS image families from Batch, view a list of VM OS images.
- Specify an image version. To use a specific version of a VM OS image, use the following format:
```
projects/IMAGE_PROJECT_ID/global/images/IMAGE_NAME
```
  Replace the following:
  - IMAGE_PROJECT_ID: the project ID of the project that contains the image. For example, for all Batch images, specify batch-custom-image.
  - IMAGE_NAME: the image name, which represents a specific version of a VM OS image. For example, to see all the VM OS image versions from Batch, view a list of VM OS images.
BOOT_DISK_TYPE: the disk type of the boot disk, either pd-standard, pd-balanced, pd-ssd, or pd-extreme. The default disk type for boot disks is pd-balanced.
BOOT_DISK_SIZE: the size of the new persistent boot disk in GB. This value must comply with all the boot disk size requirements.

What's next

If you have issues creating or running a job, see Troubleshooting.
Learn more about boot disks and VM OS images.
Learn more about creating a job.
Learn how to view jobs and tasks.