建立及執行使用儲存空間磁碟區的工作

本文說明如何建立及執行使用一或多個外部儲存空間磁碟區的 Batch 工作。外部儲存空間選項包括新的或現有的永久磁碟、新的本機 SSD、現有的 Cloud Storage 值區,以及現有的網路檔案系統 (NFS),例如 Filestore 檔案共用。

無論是否新增外部儲存空間磁碟區,每個作業的 Compute Engine VM 都會有開機磁碟,提供作業的作業系統 (OS) 映像檔和指令儲存空間。如要瞭解如何設定作業的開機磁碟,請參閱「VM OS 環境總覽」。

事前準備

  1. 如果您從未使用過 Batch,請參閱「開始使用 Batch」,並完成專案和使用者的必要條件,啟用 Batch。
  2. 如要取得建立作業所需的權限,請要求管理員授予下列 IAM 角色:

    如要進一步瞭解如何授予角色,請參閱「管理專案、資料夾和機構的存取權」。

    您或許還可透過自訂角色或其他預先定義的角色取得必要權限。

建立使用儲存空間磁碟區的工作

工作可選擇使用下列一或多種外部儲存空間磁碟區。如要進一步瞭解所有類型的儲存空間磁碟區,以及各類型的差異和限制,請參閱 Compute Engine VM 儲存空間選項說明文件。

如要允許工作使用每個儲存空間磁碟區,請將磁碟區納入工作定義,並在可執行檔中指定掛接路徑 (mountPath)。如要瞭解如何建立使用儲存空間磁碟區的工作,請參閱下列一或多個章節:

使用永久磁碟

使用永久磁碟的工作有以下限制:

  • 所有永久磁碟:查看所有永久磁碟的限制

  • 新永久磁碟與現有永久磁碟:工作中的每個永久磁碟可以是新的 (在工作中定義並建立),也可以是現有的 (已在專案中建立並在工作中指定)。如要使用永久磁碟,必須格式化並掛接至作業的 VM,且 VM 必須與永久磁碟位於相同位置。Batch 會掛接您在工作中加入的任何永久磁碟,並格式化任何新的永久磁碟,但您必須格式化並卸載工作要使用的任何現有永久磁碟。

    支援的位置選項格式選項掛接選項會因永久磁碟是新建立還是現有而有所不同,詳情請參閱下表:

    新的永久磁碟 現有的永久磁碟
    格式選項

    系統會自動將永久磁碟格式化為 ext4 檔案系統

    您必須格式化永久磁碟,才能使用 ext4 檔案系統,然後才能將磁碟用於作業。

    掛架選項

    支援所有選項

    系統支援所有選項,但撰寫除外。這是因為多重寫入模式的限制。

    您必須先從所有已連結的 VM 中卸離永久磁碟,才能用於工作。

    位置選項

    您只能建立區域永久磁碟

    你可以為工作選取任何地點。永久磁碟會在專案執行的區域中建立。

    您可以選取區域和地區永久磁碟


    您必須將工作的位置 (或如果已指定,則僅限工作的允許位置) 設為僅包含工作所有永久磁碟的位置。舉例來說,如果是區域永久磁碟,工作的位置必須是磁碟的區域;如果是地區永久磁碟,工作的位置必須是磁碟的地區,或是 (如果指定區域) 地區永久磁碟所在的一或兩個特定區域。

  • 執行個體範本:如要在建立這項工作時使用 VM 執行個體範本,您必須在執行個體範本中附加這項工作的任何永久磁碟。否則,如不想使用執行個體範本,您必須直接在工作定義中附加任何永久磁碟。

您可以使用Google Cloud 控制台、gcloud CLI、Batch API、C++、Go、Java、Node.js 或 Python,建立使用永久磁碟的工作。

控制台

以下範例使用 Google Cloud 控制台建立作業,執行指令碼從 us-central1-a 區域的現有區域永久磁碟讀取檔案。範例指令碼假設作業已具備區域永久磁碟,且該磁碟的根目錄中含有名為 example.txt 的文字檔。

選用:建立範例區域永久磁碟

如要建立可用來執行範例指令碼的區域永久磁碟,請先執行下列步驟,再建立作業:

  1. 將名為 example-disk 的新空白永久磁碟連接至 us-central1-a 可用區中的 Linux VM,然後在 VM 上執行指令,格式化及掛接磁碟。如需操作說明,請參閱「將永久磁碟新增至 VM」。

    請勿中斷與 VM 的連線。

  2. 如要在永久磁碟上建立 example.txt,請在 VM 上執行下列指令:

    1. 如要將目前的工作目錄變更為永久磁碟的根目錄,請輸入下列指令:

      cd VM_MOUNT_PATH
      

      VM_MOUNT_PATH 替換為上一步中,將永久磁碟掛接至這個 VM 的目錄路徑,例如 /mnt/disks/example-disk

    2. 按下 Enter 鍵。

    3. 如要建立及定義名為 example.txt 的檔案,請輸入下列指令:

      cat > example.txt
      
    4. 按下 Enter 鍵。

    5. 輸入檔案內容。例如,輸入 Hello world!

    6. 如要儲存檔案,請按 Ctrl+D 鍵 (或 macOS 上的 Command+D 鍵)。

    完成後,即可中斷與 VM 的連線。

  3. 從 VM 卸離永久磁碟。

    • 如果不再需要 VM,可以刪除 VM,系統會自動卸離永久磁碟。

    • 否則請卸離永久磁碟。如需操作說明,請參閱「卸離和重新連結開機磁碟」,並卸離 example-disk 永久磁碟,而非 VM 的開機磁碟。

建立使用現有區域永久磁碟的工作

如要使用Google Cloud 控制台建立使用現有區域永久磁碟的作業,請按照下列步驟操作:

  1. 前往 Google Cloud 控制台的「Job list」(工作清單) 頁面。

    前往工作清單

  2. 按一下「 Create」(建立)。「Create batch job」(建立批次工作) 頁面隨即開啟。左側窗格會選取「工作詳細資料」頁面。

  3. 設定「工作詳細資料」頁面:

    1. 選用:在「Job name」(工作名稱) 欄位中,自訂工作名稱。

      例如輸入 example-disk-job

    2. 設定「工作詳細資料」部分:

      1. 在「新增可執行項目」視窗中,為這項工作新增至少一個指令碼或容器。

        舉例來說,如要執行指令碼,列印名為 example.txt 的檔案內容,該檔案位於這項作業使用的永久磁碟根目錄中,請執行下列操作:

        1. 勾選「指令碼」核取方塊。畫面上會顯示文字方塊。

        2. 在文字方塊中輸入下列指令碼:

          echo "Here is the content of the example.txt file in the persistent disk."
          cat MOUNT_PATH/example.txt
          

          MOUNT_PATH 替換為您打算將永久磁碟掛接到這項工作 VM 的路徑,例如 /mnt/disks/example-disk

        3. 按一下 [完成]

      2. 在「工作數」欄位中,輸入這項工作的工作數。

        例如,輸入 1 (預設值)。

      3. 在「Parallelism」(平行處理) 欄位中,輸入要並行執行的工作數量。

        例如,輸入 1 (預設值)。

  4. 設定「資源規格」頁面:

    1. 在左側窗格中,按一下「資源規格」。 「資源規格」頁面隨即開啟。

    2. 選取這項工作的地點。 如要使用現有的區域永久磁碟,工作 VM 必須位於同一個區域。

      1. 在「Region」(區域) 欄位中選取區域。

        舉例來說,如要使用範例區域永久磁碟,請選取 us-central1 (Iowa) (預設)。

      2. 在「Zone」(可用區) 欄位中,選取可用區。

        例如選取 us-central1-a (Iowa)

  5. 設定「其他設定」頁面:

    1. 按一下左側窗格中的「其他設定」。 「其他設定」頁面隨即開啟。

    2. 針對要掛接至這項工作的每個現有區域永久磁碟,執行下列操作:

      1. 在「儲存空間磁碟區」部分中,按一下「新增磁碟區」。 畫面上會顯示「New volume」(新增磁碟區) 視窗。

      2. 在「New volume」(新增卷宗) 視窗中,執行下列操作:

        1. 在「Volume type」(磁碟區類型) 區段中,選取「Persistent disk」(永久磁碟) (預設)。

        2. 在「Disk」(磁碟) 清單中,選取要掛接至這項工作的現有區域永久磁碟。磁碟必須與這項工作位於相同區域。

          舉例來說,選取您準備好的現有區域永久磁碟,該磁碟位於 us-central1-a 區域,且包含 example.txt 檔案。

        3. 選用:如要重新命名這個區域永久磁碟,請按照下列步驟操作:

          1. 選取「自訂裝置名稱」

          2. 在「裝置名稱」欄位中,輸入磁碟的新名稱。

        4. 在「掛接路徑」欄位中,輸入這個永久磁碟的掛接路徑 (MOUNT_PATH):

          例如輸入以下內容:

          /mnt/disks/EXISTING_PERSISTENT_DISK_NAME
          

          EXISTING_PERSISTENT_DISK_NAME 替換為磁碟名稱。如果您已重新命名區域永久磁碟,請使用新名稱。

          例如,將 EXISTING_PERSISTENT_DISK_NAME 替換為 example-disk

        5. 按一下 [完成]

  6. 選用:設定這項工作的其他欄位

  7. 選用:如要檢查工作設定,請在左側窗格中按一下「預覽」

  8. 點選「建立」

「Job details」(工作詳細資料) 頁面會顯示您建立的工作。

gcloud

以下範例使用 gcloud CLI 建立作業,附加並掛接現有永久磁碟和新永久磁碟。這項工作有 3 項任務,每項任務都會執行指令碼,在名為 output_task_TASK_INDEX.txt 的新永久磁碟中建立檔案,其中 TASK_INDEX 是每項任務的索引:0、1 和 2。

如要使用 gcloud CLI 建立使用永久磁碟的工作,請使用 gcloud batch jobs submit 指令。在作業的 JSON 設定檔中,於 instances 欄位指定永久磁碟,並在 volumes 欄位掛接永久磁碟。

  1. 建立 JSON 檔案。

    • 如果這項作業未使用執行個體範本,請建立含有下列內容的 JSON 檔案:

      {
          "allocationPolicy": {
              "instances": [
                  {
                      "policy": {
                          "disks": [
                              {
                                  "deviceName": "EXISTING_PERSISTENT_DISK_NAME",
                                  "existingDisk": "projects/PROJECT_ID/EXISTING_PERSISTENT_DISK_LOCATION/disks/EXISTING_PERSISTENT_DISK_NAME"
                              },
                              {
                                  "newDisk": {
                                      "sizeGb": NEW_PERSISTENT_DISK_SIZE,
                                      "type": "NEW_PERSISTENT_DISK_TYPE"
                                  },
                                  "deviceName": "NEW_PERSISTENT_DISK_NAME"
                              }
                          ]
                      }
                  }
              ],
              "location": {
                  "allowedLocations": [
                      "EXISTING_PERSISTENT_DISK_LOCATION"
                  ]
              }
          },
          "taskGroups": [
              {
                  "taskSpec": {
                      "runnables": [
                          {
                              "script": {
                                  "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/disks/NEW_PERSISTENT_DISK_NAME/output_task_${BATCH_TASK_INDEX}.txt"
                              }
                          }
                      ],
                      "volumes": [
                          {
                              "deviceName": "NEW_PERSISTENT_DISK_NAME",
                              "mountPath": "/mnt/disks/NEW_PERSISTENT_DISK_NAME",
                              "mountOptions": "rw,async"
                          },
                          {
      
                              "deviceName": "EXISTING_PERSISTENT_DISK_NAME",
                              "mountPath": "/mnt/disks/EXISTING_PERSISTENT_DISK_NAME"
                          }
                      ]
                  },
                  "taskCount":3
              }
          ],
          "logsPolicy": {
              "destination": "CLOUD_LOGGING"
          }
      }
      

      更改下列內容:

      • PROJECT_ID:專案的專案 ID
      • EXISTING_PERSISTENT_DISK_NAME:現有永久磁碟的名稱。
      • EXISTING_PERSISTENT_DISK_LOCATION:現有永久磁碟的位置。如果是現有的區域永久磁碟,作業位置必須是磁碟的可用區;如果是現有的地區永久磁碟,作業位置必須是磁碟的地區,或是指定地區永久磁碟所在的特定可用區 (一或兩個)。如未指定任何現有的永久磁碟,可以選取任何位置。進一步瞭解 allowedLocations 欄位
      • NEW_PERSISTENT_DISK_SIZE:新永久磁碟的大小 (以 GB 為單位)。允許的大小取決於永久磁碟類型,但通常下限為 10 GB (10),上限為 64 TB (64000)。
      • NEW_PERSISTENT_DISK_TYPE:新永久磁碟的磁碟類型,可以是 pd-standardpd-balancedpd-ssdpd-extreme。非開機永久磁碟的預設磁碟類型為 pd-standard
      • NEW_PERSISTENT_DISK_NAME:新永久磁碟的名稱。
    • 如果您使用 VM 執行個體範本執行這項工作,請建立如先前所示的 JSON 檔案,但請將 instances 欄位替換為下列內容:

      "instances": [
          {
              "instanceTemplate": "INSTANCE_TEMPLATE_NAME"
          }
      ],
      

      其中 INSTANCE_TEMPLATE_NAME 是這項工作的執行個體範本名稱。如果工作使用永久磁碟,這個執行個體範本必須定義並附加您要工作使用的永久磁碟。以這個範例來說,範本必須定義並連結名為 NEW_PERSISTENT_DISK_NAME 的新永久磁碟,以及連結名為 EXISTING_PERSISTENT_DISK_NAME 的現有永久磁碟。

  2. 執行下列指令:

    gcloud batch jobs submit JOB_NAME \
      --location LOCATION \
      --config JSON_CONFIGURATION_FILE
    

    更改下列內容:

    • JOB_NAME:作業名稱。

    • LOCATION:工作地點

    • JSON_CONFIGURATION_FILE:JSON 檔案的路徑,內含作業的設定詳細資料。

API

下列範例使用 Batch API 建立工作,附加並掛接現有永久磁碟和新永久磁碟。這項工作有 3 項任務,每項任務都會執行指令碼,在名為 output_task_TASK_INDEX.txt 的新永久磁碟中建立檔案,其中 TASK_INDEX 是每項任務的索引:0、1 和 2。

如要使用 Batch API 建立使用永久磁碟的工作,請使用 jobs.create 方法。在要求中,於 instances 欄位指定永久磁碟,並在 volumes 欄位中掛接永久磁碟。

  • 如果這項工作未使用執行個體範本,請提出下列要求:

    POST https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/jobs?job_id=JOB_NAME
    
    {
        "allocationPolicy": {
            "instances": [
                {
                    "policy": {
                        "disks": [
                            {
                                "deviceName": "EXISTING_PERSISTENT_DISK_NAME",
                                "existingDisk": "projects/PROJECT_ID/EXISTING_PERSISTENT_DISK_LOCATION/disks/EXISTING_PERSISTENT_DISK_NAME"
                            },
                            {
                                "newDisk": {
                                    "sizeGb": NEW_PERSISTENT_DISK_SIZE,
                                    "type": "NEW_PERSISTENT_DISK_TYPE"
                                },
                                "deviceName": "NEW_PERSISTENT_DISK_NAME"
                            }
                        ]
                    }
                }
            ],
            "location": {
                "allowedLocations": [
                    "EXISTING_PERSISTENT_DISK_LOCATION"
                ]
            }
        },
        "taskGroups": [
            {
                "taskSpec": {
                    "runnables": [
                        {
                            "script": {
                                "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/disks/NEW_PERSISTENT_DISK_NAME/output_task_${BATCH_TASK_INDEX}.txt"
                            }
                        }
                    ],
                    "volumes": [
                        {
                            "deviceName": "NEW_PERSISTENT_DISK_NAME",
                            "mountPath": "/mnt/disks/NEW_PERSISTENT_DISK_NAME",
                            "mountOptions": "rw,async"
                        },
                        {
    
                            "deviceName": "EXISTING_PERSISTENT_DISK_NAME",
                            "mountPath": "/mnt/disks/EXISTING_PERSISTENT_DISK_NAME"
                        }
                    ]
                },
                "taskCount":3
            }
        ],
        "logsPolicy": {
            "destination": "CLOUD_LOGGING"
        }
    }
    

    更改下列內容:

    • PROJECT_ID:專案的專案 ID
    • LOCATION:工作地點
    • JOB_NAME:作業名稱。
    • EXISTING_PERSISTENT_DISK_NAME:現有永久磁碟的名稱。
    • EXISTING_PERSISTENT_DISK_LOCATION:現有永久磁碟的位置。對於每個現有的區域永久磁碟,作業的位置必須是磁碟的可用區;對於每個現有的區域永久磁碟,作業的位置必須是磁碟的區域,或是 (如果指定可用區) 區域永久磁碟所在的一或兩個特定可用區。如果您未指定任何現有永久磁碟,可以選取任何位置。進一步瞭解 allowedLocations 欄位
    • NEW_PERSISTENT_DISK_SIZE:新永久磁碟的大小 (以 GB 為單位)。允許的大小取決於永久磁碟類型,但通常下限為 10 GB (10),上限為 64 TB (64000)。
    • NEW_PERSISTENT_DISK_TYPE:新永久磁碟的磁碟類型,可以是 pd-standardpd-balancedpd-ssdpd-extreme。非開機永久磁碟的預設磁碟類型為 pd-standard
    • NEW_PERSISTENT_DISK_NAME:新永久磁碟的名稱。
  • 如果您使用 VM 執行個體範本執行這項工作,請建立如先前所示的 JSON 檔案,但請將 instances 欄位替換為下列內容:

    "instances": [
        {
            "instanceTemplate": "INSTANCE_TEMPLATE_NAME"
        }
    ],
    ...
    

    其中 INSTANCE_TEMPLATE_NAME 是這項工作的執行個體範本名稱。如果工作使用永久磁碟,這個執行個體範本必須定義並附加您要工作使用的永久磁碟。以這個範例來說,範本必須定義並連結名為 NEW_PERSISTENT_DISK_NAME 的新永久磁碟,以及連結名為 EXISTING_PERSISTENT_DISK_NAME 的現有永久磁碟。

C++

如要使用 C++ 適用的 Cloud 用戶端程式庫建立使用新或現有永久磁碟的 Batch 工作,請使用 CreateJob 函式,並加入下列項目:

  • 如要將永久磁碟連結至作業的 VM,請加入下列其中一項:
    • 如果這項工作未使用 VM 執行個體範本,請使用 set_remote_path 方法。
    • 如果這項工作使用 VM 執行個體範本,請使用 set_instance_template 方法。
  • 如要將永久磁碟掛接至工作,請搭配 deviceNamemountPath 欄位使用 volumes 欄位。如果是新的永久磁碟,請使用 mountOptions 欄位啟用寫入功能。

如需類似用途的程式碼範例,請參閱「使用 Cloud Storage bucket」。

Go

如要使用 Cloud Client Libraries for Go 建立使用新或現有永久磁碟的 Batch 工作,請使用 CreateJob 函式,並加入下列項目:

import (
	"context"
	"fmt"
	"io"

	batch "cloud.google.com/go/batch/apiv1"
	"cloud.google.com/go/batch/apiv1/batchpb"
	durationpb "google.golang.org/protobuf/types/known/durationpb"
)

// Creates and runs a job with persistent disk
func createJobWithPD(w io.Writer, projectID, jobName, pdName string) error {
	// jobName := job-name
	// pdName := disk-name
	ctx := context.Background()
	batchClient, err := batch.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("batchClient error: %w", err)
	}
	defer batchClient.Close()

	runn := &batchpb.Runnable{
		Executable: &batchpb.Runnable_Script_{
			Script: &batchpb.Runnable_Script{
				Command: &batchpb.Runnable_Script_Text{
					Text: "echo Hello world from script 1 for task ${BATCH_TASK_INDEX}",
				},
			},
		},
	}
	volume := &batchpb.Volume{
		MountPath: fmt.Sprintf("/mnt/disks/%v", pdName),
		Source: &batchpb.Volume_DeviceName{
			DeviceName: pdName,
		},
	}

	// The disk type of the new persistent disk, either pd-standard,
	// pd-balanced, pd-ssd, or pd-extreme. For Batch jobs, the default is pd-balanced
	disk := &batchpb.AllocationPolicy_Disk{
		Type:   "pd-balanced",
		SizeGb: 10,
	}

	taskSpec := &batchpb.TaskSpec{
		ComputeResource: &batchpb.ComputeResource{
			// CpuMilli is milliseconds per cpu-second. This means the task requires 1 CPU.
			CpuMilli:  1000,
			MemoryMib: 16,
		},
		MaxRunDuration: &durationpb.Duration{
			Seconds: 3600,
		},
		MaxRetryCount: 2,
		Runnables:     []*batchpb.Runnable{runn},
		Volumes:       []*batchpb.Volume{volume},
	}

	taskGroups := []*batchpb.TaskGroup{
		{
			TaskCount: 4,
			TaskSpec:  taskSpec,
		},
	}

	labels := map[string]string{"env": "testing", "type": "container"}

	// Policies are used to define on what kind of virtual machines the tasks will run on.
	// Read more about local disks here: https://cloud.google.com/compute/docs/disks/persistent-disks
	allocationPolicy := &batchpb.AllocationPolicy{
		Instances: []*batchpb.AllocationPolicy_InstancePolicyOrTemplate{{
			PolicyTemplate: &batchpb.AllocationPolicy_InstancePolicyOrTemplate_Policy{
				Policy: &batchpb.AllocationPolicy_InstancePolicy{
					MachineType: "n1-standard-1",
					Disks: []*batchpb.AllocationPolicy_AttachedDisk{
						{
							Attached: &batchpb.AllocationPolicy_AttachedDisk_NewDisk{
								NewDisk: disk,
							},
							DeviceName: pdName,
						},
					},
				},
			},
		}},
	}

	// We use Cloud Logging as it's an out of the box available option
	logsPolicy := &batchpb.LogsPolicy{
		Destination: batchpb.LogsPolicy_CLOUD_LOGGING,
	}

	job := &batchpb.Job{
		Name:             jobName,
		TaskGroups:       taskGroups,
		AllocationPolicy: allocationPolicy,
		Labels:           labels,
		LogsPolicy:       logsPolicy,
	}

	request := &batchpb.CreateJobRequest{
		Parent: fmt.Sprintf("projects/%s/locations/%s", projectID, "us-central1"),
		JobId:  jobName,
		Job:    job,
	}

	created_job, err := batchClient.CreateJob(ctx, request)
	if err != nil {
		return fmt.Errorf("unable to create job: %w", err)
	}

	fmt.Fprintf(w, "Job created: %v\n", created_job)
	return nil
}

Java

如要使用 Java 適用的 Cloud 用戶端程式庫建立使用新或現有永久磁碟的 Batch 工作,請使用 CreateJobRequest 類別,並加入下列項目:

舉例來說,請使用下列程式碼範例:


import com.google.cloud.batch.v1.AllocationPolicy;
import com.google.cloud.batch.v1.AllocationPolicy.AttachedDisk;
import com.google.cloud.batch.v1.AllocationPolicy.Disk;
import com.google.cloud.batch.v1.AllocationPolicy.InstancePolicy;
import com.google.cloud.batch.v1.AllocationPolicy.InstancePolicyOrTemplate;
import com.google.cloud.batch.v1.AllocationPolicy.LocationPolicy;
import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.Runnable.Script;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.cloud.batch.v1.Volume;
import com.google.common.collect.Lists;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreatePersistentDiskJob {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "europe-central2";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "JOB_NAME";
    // The size of the new persistent disk in GB.
    // The allowed sizes depend on the type of persistent disk,
    // but the minimum is often 10 GB (10) and the maximum is often 64 TB (64000).
    int diskSize = 10;
    // The name of the new persistent disk.
    String newPersistentDiskName = "DISK-NAME";
    // The name of an existing persistent disk.
    String existingPersistentDiskName = "EXISTING-DISK-NAME";
    // The location of an existing persistent disk. For more info :
    // https://cloud.google.com/batch/docs/create-run-job-storage#gcloud
    String location = "regions/us-central1";
    // The disk type of the new persistent disk, either pd-standard,
    // pd-balanced, pd-ssd, or pd-extreme. For Batch jobs, the default is pd-balanced.
    String newDiskType = "pd-balanced";

    createPersistentDiskJob(projectId, region, jobName, newPersistentDiskName,
            diskSize, existingPersistentDiskName, location, newDiskType);
  }

  // Creates a job that attaches and mounts an existing persistent disk and a new persistent disk
  public static Job createPersistentDiskJob(String projectId, String region, String jobName,
                                            String newPersistentDiskName, int diskSize,
                                            String existingPersistentDiskName,
                                            String location, String newDiskType)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {
      // Define what will be done as part of the job.
      String text = "echo Hello world from task ${BATCH_TASK_INDEX}. "
              + ">> /mnt/disks/NEW_PERSISTENT_DISK_NAME/output_task_${BATCH_TASK_INDEX}.txt";
      Runnable runnable =
          Runnable.newBuilder()
              .setScript(
                  Script.newBuilder()
                      .setText(text)
                      // You can also run a script from a file. Just remember, that needs to be a
                      // script that's already on the VM that will be running the job.
                      // Using setText() and setPath() is mutually exclusive.
                      // .setPath("/tmp/test.sh")
                      .build())
              .build();

      TaskSpec task = TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addAllVolumes(volumes(newPersistentDiskName, existingPersistentDiskName))
              .addRunnables(runnable)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder()
          .setTaskCount(3)
          .setParallelism(1)
          .setTaskSpec(task)
          .build();

      // Policies are used to define the type of virtual machines the tasks will run on.
      InstancePolicy policy = InstancePolicy.newBuilder()
              .addAllDisks(attachedDisks(newPersistentDiskName, diskSize, newDiskType,
                  projectId, location, existingPersistentDiskName))
              .build();

      AllocationPolicy allocationPolicy =
          AllocationPolicy.newBuilder()
              .addInstances(
                  InstancePolicyOrTemplate.newBuilder()
                      .setPolicy(policy))
                  .setLocation(LocationPolicy.newBuilder().addAllowedLocations(location))
              .build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              .setAllocationPolicy(allocationPolicy)
              .putLabels("env", "testing")
              .putLabels("type", "script")
              // We use Cloud Logging as it's an out-of-the-box option.
              .setLogsPolicy(
                  LogsPolicy.newBuilder().setDestination(LogsPolicy.Destination.CLOUD_LOGGING))
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }

  // Creates link to existing disk and creates configuration for new disk
  private static Iterable<AttachedDisk> attachedDisks(String newPersistentDiskName, int diskSize,
                                                      String newDiskType, String projectId,
                                                      String existingPersistentDiskLocation,
                                                      String existingPersistentDiskName) {
    AttachedDisk newDisk = AttachedDisk.newBuilder()
            .setDeviceName(newPersistentDiskName)
            .setNewDisk(Disk.newBuilder().setSizeGb(diskSize).setType(newDiskType))
            .build();

    String diskPath = String.format("projects/%s/%s/disks/%s", projectId,
            existingPersistentDiskLocation, existingPersistentDiskName);

    AttachedDisk existingDisk = AttachedDisk.newBuilder()
            .setDeviceName(existingPersistentDiskName)
            .setExistingDisk(diskPath)
            .build();

    return Lists.newArrayList(existingDisk, newDisk);
  }

  // Describes a volume and parameters for it to be mounted to a VM.
  private static Iterable<Volume> volumes(String newPersistentDiskName,
                                          String existingPersistentDiskName) {
    Volume newVolume = Volume.newBuilder()
            .setDeviceName(newPersistentDiskName)
            .setMountPath("/mnt/disks/" + newPersistentDiskName)
            .addMountOptions("rw")
            .addMountOptions("async")
            .build();

    Volume existingVolume = Volume.newBuilder()
            .setDeviceName(existingPersistentDiskName)
            .setMountPath("/mnt/disks/" + existingPersistentDiskName)
            .build();

    return Lists.newArrayList(newVolume, existingVolume);
  }
}

Node.js

如要使用 Node.js 適用的 Cloud 用戶端程式庫,建立使用新或現有永久磁碟的 Batch 工作,請使用 createJob 方法,並加入下列項目:

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'batch-create-persistent-disk-job';
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'europe-central2';
// The name of an existing persistent disk.
const existingPersistentDiskName = 'existing-persistent-disk-name';
// The name of the new persistent disk.
const newPersistentDiskName = 'new-persistent-disk-name';
// The size of the new persistent disk in GB.
// The allowed sizes depend on the type of persistent disk,
// but the minimum is often 10 GB (10) and the maximum is often 64 TB (64000).
const diskSize = 10;
// The location of an existing persistent disk. For more info :
// https://cloud.google.com/batch/docs/create-run-job-storage#gcloud
const location = 'regions/us-central1';
// The disk type of the new persistent disk, either pd-standard,
// pd-balanced, pd-ssd, or pd-extreme. For Batch jobs, the default is pd-balanced.
const newDiskType = 'pd-balanced';

// Define what will be done as part of the job.
const runnable = new batch.Runnable({
  script: new batch.Runnable.Script({
    commands: [
      '-c',
      'echo Hello world! This is task ${BATCH_TASK_INDEX}.' +
        '>> /mnt/disks/NEW_PERSISTENT_DISK_NAME/output_task_${BATCH_TASK_INDEX}.txt',
    ],
  }),
});

// Define volumes and their parameters to be mounted to a VM.
const newVolume = new batch.Volume({
  deviceName: newPersistentDiskName,
  mountPath: `/mnt/disks/${newPersistentDiskName}`,
  mountOptions: ['rw', 'async'],
});

const existingVolume = new batch.Volume({
  deviceName: existingPersistentDiskName,
  mountPath: `/mnt/disks/${existingPersistentDiskName}`,
});

const task = new batch.TaskSpec({
  runnables: [runnable],
  volumes: [newVolume, existingVolume],
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

const newDisk = new batch.AllocationPolicy.Disk({
  type: newDiskType,
  sizeGb: diskSize,
});

// Policies are used to define on what kind of virtual machines the tasks will run on.
// Read more about local disks here: https://cloud.google.com/compute/docs/disks/persistent-disks
const instancePolicy = new batch.AllocationPolicy.InstancePolicy({
  disks: [
    // Create configuration for new disk
    new batch.AllocationPolicy.AttachedDisk({
      deviceName: newPersistentDiskName,
      newDisk,
    }),
    // Create link to existing disk
    new batch.AllocationPolicy.AttachedDisk({
      existingDisk: `projects/${projectId}/${location}/disks/${existingPersistentDiskName}`,
      deviceName: existingPersistentDiskName,
    }),
  ],
});

const locationPolicy = new batch.AllocationPolicy.LocationPolicy({
  allowedLocations: [location],
});

const allocationPolicy = new batch.AllocationPolicy.InstancePolicyOrTemplate({
  instances: [{policy: instancePolicy}],
  location: locationPolicy,
});

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  labels: {env: 'testing', type: 'script'},
  allocationPolicy,
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});
// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchPersistentDiskJob() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchPersistentDiskJob();

Python

如要使用 Python 適用的 Cloud 用戶端程式庫建立 Batch 工作,並使用新的或現有的永久磁碟,請使用 CreateJob 函式,並加入下列項目:

  • 如要將永久磁碟連結至作業的 VM,請加入下列其中一項:
  • 如要將永久磁碟掛接到工作,請使用 Volume 類別搭配 device_name 屬性和 mount_path 屬性。如果是新的永久磁碟,請一併使用 mount_options 屬性啟用寫入功能。

舉例來說,請使用下列程式碼範例:

from google.cloud import batch_v1


def create_with_pd_job(
    project_id: str,
    region: str,
    job_name: str,
    disk_name: str,
    zone: str,
    existing_disk_name=None,
) -> batch_v1.Job:
    """
    This method shows how to create a sample Batch Job that will run
    a simple command on Cloud Compute instances with mounted persistent disk.

    Args:
        project_id: project ID or project number of the Cloud project you want to use.
        region: name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
        job_name: the name of the job that will be created.
            It needs to be unique for each project and region pair.
        disk_name: name of the disk to be mounted for your Job.
        existing_disk_name(optional): existing disk name, which you want to attach to a job

    Returns:
        A job object representing the job created.
    """
    client = batch_v1.BatchServiceClient()

    # Define what will be done as part of the job.
    task = batch_v1.TaskSpec()
    runnable = batch_v1.Runnable()
    runnable.script = batch_v1.Runnable.Script()
    runnable.script.text = (
        "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/disks/"
        + disk_name
        + "/output_task_${BATCH_TASK_INDEX}.txt"
    )
    task.runnables = [runnable]
    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    volume = batch_v1.Volume()
    volume.device_name = disk_name
    volume.mount_path = f"/mnt/disks/{disk_name}"
    task.volumes = [volume]

    if existing_disk_name:
        volume2 = batch_v1.Volume()
        volume2.device_name = existing_disk_name
        volume2.mount_path = f"/mnt/disks/{existing_disk_name}"
        task.volumes.append(volume2)

    # Tasks are grouped inside a job using TaskGroups.
    # Currently, it's possible to have only one task group.
    group = batch_v1.TaskGroup()
    group.task_count = 4
    group.task_spec = task

    disk = batch_v1.AllocationPolicy.Disk()
    # The disk type of the new persistent disk, either pd-standard,
    # pd-balanced, pd-ssd, or pd-extreme. For Batch jobs, the default is pd-balanced
    disk.type_ = "pd-balanced"
    disk.size_gb = 10

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # Read more about local disks here: https://cloud.google.com/compute/docs/disks/persistent-disks
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "n1-standard-1"

    attached_disk = batch_v1.AllocationPolicy.AttachedDisk()
    attached_disk.new_disk = disk
    attached_disk.device_name = disk_name
    policy.disks = [attached_disk]

    if existing_disk_name:
        attached_disk2 = batch_v1.AllocationPolicy.AttachedDisk()
        attached_disk2.existing_disk = (
            f"projects/{project_id}/zones/{zone}/disks/{existing_disk_name}"
        )
        attached_disk2.device_name = existing_disk_name
        policy.disks.append(attached_disk2)

    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy

    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    location = batch_v1.AllocationPolicy.LocationPolicy()
    location.allowed_locations = [f"zones/{zone}"]
    allocation_policy.location = location

    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy
    job.labels = {"env": "testing", "type": "script"}

    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

使用本機 SSD

使用本機 SSD 的作業有下列限制:

您可以使用 gcloud CLI、Batch API、Java 或 Python,建立使用本機 SSD 的工作。以下範例說明如何建立工作,以建立、附加及掛接本機 SSD。這項作業也有 3 項工作,每項工作都會執行指令碼,在本機 SSD 中建立名為 output_task_TASK_INDEX.txt 的檔案,其中 TASK_INDEX 是每項工作的索引:012

gcloud

如要使用 gcloud CLI 建立使用本機 SSD 的工作,請使用 gcloud batch jobs submit 指令。在作業的 JSON 設定檔中,於 instances 欄位建立並連結本機 SSD,然後在 volumes 欄位掛接本機 SSD。

  1. 建立 JSON 檔案。

    • 如果這項作業未使用執行個體範本,請建立含有下列內容的 JSON 檔案:

      {
          "allocationPolicy": {
              "instances": [
                  {
                      "policy": {
                          "machineType": MACHINE_TYPE,
                          "disks": [
                              {
                                  "newDisk": {
                                      "sizeGb": LOCAL_SSD_SIZE,
                                      "type": "local-ssd"
                                  },
                                  "deviceName": "LOCAL_SSD_NAME"
                              }
                          ]
                      }
                  }
              ]
          },
          "taskGroups": [
              {
                  "taskSpec": {
                      "runnables": [
                          {
                              "script": {
                                  "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/disks/LOCAL_SSD_NAME/output_task_${BATCH_TASK_INDEX}.txt"
                              }
                          }
                      ],
                      "volumes": [
                          {
                              "deviceName": "LOCAL_SSD_NAME",
                              "mountPath": "/mnt/disks/LOCAL_SSD_NAME",
                              "mountOptions": "rw,async"
                          }
                      ]
                  },
                  "taskCount":3
              }
          ],
          "logsPolicy": {
              "destination": "CLOUD_LOGGING"
          }
      }
      

      更改下列內容:

      • MACHINE_TYPE:工作的 VM 機器類型,可以是預先定義自訂允許的本機 SSD 數量取決於作業 VM 的機器類型。
      • LOCAL_SSD_NAME:為這項工作建立的本機 SSD 名稱。
      • LOCAL_SSD_SIZE:所有本機 SSD 的大小,以 GB 為單位。每個本機 SSD 為 375 GB,因此這個值必須是 375 GB 的倍數。舉例來說,如要使用 2 個本機 SSD,請將這個值設為 750 GB。
    • 如果您使用 VM 執行個體範本執行這項工作,請建立如先前所示的 JSON 檔案,但請將 instances 欄位替換為下列內容:

      "instances": [
          {
              "instanceTemplate": "INSTANCE_TEMPLATE_NAME"
          }
      ],
      

      其中 INSTANCE_TEMPLATE_NAME 是這項工作的執行個體範本名稱。如果工作使用本機 SSD,這個執行個體範本必須定義並連結工作要使用的本機 SSD。在本範例中,範本必須定義並連結名為 LOCAL_SSD_NAME 的本機 SSD。

  2. 執行下列指令:

    gcloud batch jobs submit JOB_NAME \
      --location LOCATION \
      --config JSON_CONFIGURATION_FILE
    

    更改下列內容:

    • JOB_NAME:作業名稱。
    • LOCATION:工作地點
    • JSON_CONFIGURATION_FILE:JSON 檔案的路徑,內含作業的設定詳細資料。

API

如要使用 Batch API 建立使用本機 SSD 的作業,請使用 jobs.create 方法。在要求中,於 instances 欄位建立並連結本機 SSD,並在 volumes 欄位掛接本機 SSD。

  • 如果這項工作未使用執行個體範本,請提出下列要求:

    POST https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/jobs?job_id=JOB_NAME
    
    {
        "allocationPolicy": {
            "instances": [
                {
                    "policy": {
                        "machineType": MACHINE_TYPE,
                        "disks": [
                            {
                                "newDisk": {
                                    "sizeGb": LOCAL_SSD_SIZE,
                                    "type": "local-ssd"
                                },
                                "deviceName": "LOCAL_SSD_NAME"
                            }
                        ]
                    }
                }
            ]
        },
        "taskGroups": [
            {
                "taskSpec": {
                    "runnables": [
                        {
                            "script": {
                                "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/disks/LOCAL_SSD_NAME/output_task_${BATCH_TASK_INDEX}.txt"
                            }
                        }
                    ],
                    "volumes": [
                        {
                            "deviceName": "LOCAL_SSD_NAME",
                            "mountPath": "/mnt/disks/LOCAL_SSD_NAME",
                            "mountOptions": "rw,async"
                        }
                    ]
                },
                "taskCount":3
            }
        ],
        "logsPolicy": {
            "destination": "CLOUD_LOGGING"
        }
    }
    

    更改下列內容:

    • PROJECT_ID:專案的專案 ID
    • LOCATION:工作地點
    • JOB_NAME:作業名稱。
    • MACHINE_TYPE:工作的 VM 機器類型,可以是預先定義自訂允許的本機 SSD 數量取決於作業 VM 的機器類型。
    • LOCAL_SSD_NAME:為這項工作建立的本機 SSD 名稱。
    • LOCAL_SSD_SIZE:所有本機 SSD 的大小,以 GB 為單位。每個本機 SSD 為 375 GB,因此這個值必須是 375 GB 的倍數。舉例來說,如要使用 2 個本機 SSD,請將這個值設為 750 GB。
  • 如果您使用 VM 執行個體範本執行這項工作,請建立如先前所示的 JSON 檔案,但請將 instances 欄位替換為下列內容:

    "instances": [
        {
            "instanceTemplate": "INSTANCE_TEMPLATE_NAME"
        }
    ],
    ...
    

    其中 INSTANCE_TEMPLATE_NAME 是這項工作的執行個體範本名稱。如果工作使用本機 SSD,這個執行個體範本必須定義並連結工作要使用的本機 SSD。在本範例中,範本必須定義並連結名為 LOCAL_SSD_NAME 的本機 SSD。

Go

import (
	"context"
	"fmt"
	"io"

	batch "cloud.google.com/go/batch/apiv1"
	"cloud.google.com/go/batch/apiv1/batchpb"
	durationpb "google.golang.org/protobuf/types/known/durationpb"
)

// Creates and runs a job with local SSD
// Note: local SSD does not guarantee Local SSD data persistence.
// More details here: https://cloud.google.com/compute/docs/disks/local-ssd#data_persistence
func createJobWithSSD(w io.Writer, projectID, jobName, ssdName string) error {
	// jobName := job-name
	// ssdName := disk-name
	ctx := context.Background()
	batchClient, err := batch.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("batchClient error: %w", err)
	}
	defer batchClient.Close()

	runn := &batchpb.Runnable{
		Executable: &batchpb.Runnable_Script_{
			Script: &batchpb.Runnable_Script{
				Command: &batchpb.Runnable_Script_Text{
					Text: "echo Hello world from script 1 for task ${BATCH_TASK_INDEX}",
				},
			},
		},
	}
	volume := &batchpb.Volume{
		MountPath: fmt.Sprintf("/mnt/disks/%v", ssdName),
		Source: &batchpb.Volume_DeviceName{
			DeviceName: ssdName,
		},
	}

	// The size of all the local SSDs in GB. Each local SSD is 375 GB,
	// so this value must be a multiple of 375 GB.
	// For example, for 2 local SSDs, set this value to 750 GB.
	disk := &batchpb.AllocationPolicy_Disk{
		Type:   "local-ssd",
		SizeGb: 375,
	}

	taskSpec := &batchpb.TaskSpec{
		ComputeResource: &batchpb.ComputeResource{
			// CpuMilli is milliseconds per cpu-second. This means the task requires 1 CPU.
			CpuMilli:  1000,
			MemoryMib: 16,
		},
		MaxRunDuration: &durationpb.Duration{
			Seconds: 3600,
		},
		MaxRetryCount: 2,
		Runnables:     []*batchpb.Runnable{runn},
		Volumes:       []*batchpb.Volume{volume},
	}

	taskGroups := []*batchpb.TaskGroup{
		{
			TaskCount: 4,
			TaskSpec:  taskSpec,
		},
	}

	labels := map[string]string{"env": "testing", "type": "container"}

	allocationPolicy := &batchpb.AllocationPolicy{
		Instances: []*batchpb.AllocationPolicy_InstancePolicyOrTemplate{{
			PolicyTemplate: &batchpb.AllocationPolicy_InstancePolicyOrTemplate_Policy{
				Policy: &batchpb.AllocationPolicy_InstancePolicy{
					// The allowed number of local SSDs depends on the machine type for your job's VMs.
					// In this case, we tell the system to use "n1-standard-1" machine type, which require to attach local ssd manually.
					// Read more about local disks here: https://cloud.google.com/compute/docs/disks/local-ssd#lssd_disk_options
					MachineType: "n1-standard-1",
					Disks: []*batchpb.AllocationPolicy_AttachedDisk{
						{
							Attached: &batchpb.AllocationPolicy_AttachedDisk_NewDisk{
								NewDisk: disk,
							},
							DeviceName: ssdName,
						},
					},
				},
			},
		}},
	}

	// We use Cloud Logging as it's an out of the box available option
	logsPolicy := &batchpb.LogsPolicy{
		Destination: batchpb.LogsPolicy_CLOUD_LOGGING,
	}

	job := &batchpb.Job{
		Name:             jobName,
		TaskGroups:       taskGroups,
		AllocationPolicy: allocationPolicy,
		Labels:           labels,
		LogsPolicy:       logsPolicy,
	}

	request := &batchpb.CreateJobRequest{
		Parent: fmt.Sprintf("projects/%s/locations/%s", projectID, "us-central1"),
		JobId:  jobName,
		Job:    job,
	}

	created_job, err := batchClient.CreateJob(ctx, request)
	if err != nil {
		return fmt.Errorf("unable to create job: %w", err)
	}

	fmt.Fprintf(w, "Job created: %v\n", created_job)
	return nil
}

Java


import com.google.cloud.batch.v1.AllocationPolicy;
import com.google.cloud.batch.v1.AllocationPolicy.AttachedDisk;
import com.google.cloud.batch.v1.AllocationPolicy.Disk;
import com.google.cloud.batch.v1.AllocationPolicy.InstancePolicy;
import com.google.cloud.batch.v1.AllocationPolicy.InstancePolicyOrTemplate;
import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.Runnable.Script;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.cloud.batch.v1.Volume;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreateLocalSsdJob {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "europe-central2";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "JOB_NAME";
    // The name of a local SSD created for this job.
    String localSsdName = "SSD-NAME";
    // The machine type, which can be predefined or custom, of the job's VMs.
    // The allowed number of local SSDs depends on the machine type
    // for your job's VMs are listed on: https://cloud.google.com/compute/docs/disks#localssds
    String machineType = "c3d-standard-8-lssd";
    // The size of all the local SSDs in GB. Each local SSD is 375 GB,
    // so this value must be a multiple of 375 GB.
    // For example, for 2 local SSDs, set this value to 750 GB.
    int ssdSize = 375;

    createLocalSsdJob(projectId, region, jobName, localSsdName, ssdSize, machineType);
  }

  // Create a job that uses local SSDs
  public static Job createLocalSsdJob(String projectId, String region, String jobName,
                                      String localSsdName, int ssdSize, String machineType)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {
      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setScript(
                  Script.newBuilder()
                      .setText(
                          "echo Hello world! This is task ${BATCH_TASK_INDEX}. "
                                  + "This job has a total of ${BATCH_TASK_COUNT} tasks.")
                      // You can also run a script from a file. Just remember, that needs to be a
                      // script that's already on the VM that will be running the job.
                      // Using setText() and setPath() is mutually exclusive.
                      // .setPath("/tmp/test.sh")
                      .build())
              .build();

      Volume volume = Volume.newBuilder()
          .setDeviceName(localSsdName)
          .setMountPath("/mnt/disks/" + localSsdName)
          .addMountOptions("rw")
          .addMountOptions("async")
          .build();

      TaskSpec task = TaskSpec.newBuilder()
          // Jobs can be divided into tasks. In this case, we have only one task.
          .addVolumes(volume)
          .addRunnables(runnable)
          .setMaxRetryCount(2)
          .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
          .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder()
          .setTaskCount(3)
          .setParallelism(1)
          .setTaskSpec(task)
          .build();

      // Policies are used to define on what kind of virtual machines the tasks will run on.
      InstancePolicy policy = InstancePolicy.newBuilder()
          .setMachineType(machineType)
          .addDisks(AttachedDisk.newBuilder()
              .setDeviceName(localSsdName)
              // For example, local SSD uses type "local-ssd".
              // Persistent disks and boot disks use "pd-balanced", "pd-extreme", "pd-ssd"
              // or "pd-standard".
              .setNewDisk(Disk.newBuilder().setSizeGb(ssdSize).setType("local-ssd")))
          .build();

      AllocationPolicy allocationPolicy =
          AllocationPolicy.newBuilder()
              .addInstances(
                  InstancePolicyOrTemplate.newBuilder()
                      .setPolicy(policy)
                      .build())
              .build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              .setAllocationPolicy(allocationPolicy)
              .putLabels("env", "testing")
              .putLabels("type", "script")
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(
                  LogsPolicy.newBuilder().setDestination(LogsPolicy.Destination.CLOUD_LOGGING))
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }
}

Node.js

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'europe-central2';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'batch-local-ssd-job';
// The name of a local SSD created for this job.
const localSsdName = 'ssd-name';
// The machine type, which can be predefined or custom, of the job's VMs.
// The allowed number of local SSDs depends on the machine type
// for your job's VMs are listed on: https://cloud.google.com/compute/docs/disks#localssds
const machineType = 'c3d-standard-8-lssd';
// The size of all the local SSDs in GB. Each local SSD is 375 GB,
// so this value must be a multiple of 375 GB.
// For example, for 2 local SSDs, set this value to 750 GB.
const ssdSize = 375;

// Define what will be done as part of the job.
const runnable = new batch.Runnable({
  script: new batch.Runnable.Script({
    commands: [
      '-c',
      'echo Hello world! This is task ${BATCH_TASK_INDEX}. This job has a total of ${BATCH_TASK_COUNT} tasks.',
    ],
  }),
});

const task = new batch.TaskSpec({
  runnables: [runnable],
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

// Policies are used to define on what kind of virtual machines the tasks will run on.
const instancePolicy = new batch.AllocationPolicy.InstancePolicy({
  machineType,
  disks: [
    new batch.AllocationPolicy.AttachedDisk({
      deviceName: localSsdName,
      // For example, local SSD uses type "local-ssd".
      // Persistent disks and boot disks use "pd-balanced", "pd-extreme", "pd-ssd"
      // or "pd-standard".
      newDisk: new batch.AllocationPolicy.AttachedDisk({
        type: 'local-ssd',
        sizeGb: ssdSize,
      }),
    }),
  ],
});

const allocationPolicy = new batch.AllocationPolicy.InstancePolicyOrTemplate({
  instances: [{policy: instancePolicy}],
});

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  labels: {env: 'testing', type: 'script'},
  allocationPolicy,
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});
// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchGPUJob() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchGPUJob();

Python

from google.cloud import batch_v1


def create_local_ssd_job(
    project_id: str, region: str, job_name: str, ssd_name: str
) -> batch_v1.Job:
    """
    This method shows how to create a sample Batch Job that will run
    a simple command on Cloud Compute instances with mounted local SSD.
    Note: local SSD does not guarantee Local SSD data persistence.
    More details here: https://cloud.google.com/compute/docs/disks/local-ssd#data_persistence

    Args:
        project_id: project ID or project number of the Cloud project you want to use.
        region: name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
        job_name: the name of the job that will be created.
            It needs to be unique for each project and region pair.
        ssd_name: name of the local ssd to be mounted for your Job.

    Returns:
        A job object representing the job created.
    """
    client = batch_v1.BatchServiceClient()

    # Define what will be done as part of the job.
    task = batch_v1.TaskSpec()
    runnable = batch_v1.Runnable()
    runnable.script = batch_v1.Runnable.Script()
    runnable.script.text = "echo Hello world! This is task ${BATCH_TASK_INDEX}. This job has a total of ${BATCH_TASK_COUNT} tasks."
    task.runnables = [runnable]
    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    volume = batch_v1.Volume()
    volume.device_name = ssd_name
    volume.mount_path = f"/mnt/disks/{ssd_name}"
    task.volumes = [volume]

    # Tasks are grouped inside a job using TaskGroups.
    # Currently, it's possible to have only one task group.
    group = batch_v1.TaskGroup()
    group.task_count = 4
    group.task_spec = task

    disk = batch_v1.AllocationPolicy.Disk()
    disk.type_ = "local-ssd"
    # The size of all the local SSDs in GB. Each local SSD is 375 GB,
    # so this value must be a multiple of 375 GB.
    # For example, for 2 local SSDs, set this value to 750 GB.
    disk.size_gb = 375
    assert disk.size_gb % 375 == 0

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # The allowed number of local SSDs depends on the machine type for your job's VMs.
    # In this case, we tell the system to use "n1-standard-1" machine type, which require to attach local ssd manually.
    # Read more about local disks here: https://cloud.google.com/compute/docs/disks/local-ssd#lssd_disk_options
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "n1-standard-1"

    attached_disk = batch_v1.AllocationPolicy.AttachedDisk()
    attached_disk.new_disk = disk
    attached_disk.device_name = ssd_name
    policy.disks = [attached_disk]

    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy

    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy
    job.labels = {"env": "testing", "type": "script"}
    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

使用 Cloud Storage 值區

如要建立使用現有 Cloud Storage bucket 的工作,請選取下列其中一種方法:

  • 建議:在工作定義中指定值區,直接將值區掛接到工作的 VM,如本節所示。工作執行時,系統會使用 Cloud Storage FUSE,自動將 bucket 掛接至工作 VM。
  • 使用 gcloud CLI 或 Cloud Storage API 適用的用戶端程式庫,建立可直接存取 Cloud Storage bucket 的工作。如要瞭解如何直接從 VM 存取 Cloud Storage 值區,請參閱 Compute Engine 說明文件,瞭解如何從 Cloud Storage 值區寫入及讀取資料

建立使用值區的工作前,請先建立值區或找出現有值區。詳情請參閱「建立值區」和「列出值區」。

您可以使用Google Cloud 控制台、gcloud CLI、Batch API、C++、Go、Java、Node.js 或 Python,建立使用 Cloud Storage 值區的工作。

下列範例說明如何建立工作,並掛接 Cloud Storage 值區。這項工作也有 3 個工作,每個工作都會執行指令碼,在名為 output_task_TASK_INDEX.txt 的 Bucket 中建立檔案,其中 TASK_INDEX 是每個工作的索引:012

控制台

如要使用Google Cloud 控制台建立使用 Cloud Storage bucket 的工作,請按照下列步驟操作:

  1. 前往 Google Cloud 控制台的「Job list」(工作清單) 頁面。

    前往工作清單

  2. 按一下「 Create」(建立)。「Create batch job」(建立批次工作) 頁面隨即開啟。左側窗格會選取「工作詳細資料」頁面。

  3. 設定「工作詳細資料」頁面:

    1. 選用:在「Job name」(工作名稱) 欄位中,自訂工作名稱。

      例如輸入 example-bucket-job

    2. 設定「工作詳細資料」部分:

      1. 在「新增可執行項目」視窗中,為這項工作新增至少一個指令碼或容器。

        舉例來說,請執行下列操作:

        1. 勾選「指令碼」核取方塊。畫面上會顯示文字方塊。

        2. 在文字方塊中輸入下列指令碼:

          echo Hello world from task ${BATCH_TASK_INDEX}. >> MOUNT_PATH/output_task_${BATCH_TASK_INDEX}.txt
          

          請將 MOUNT_PATH 改成掛接路徑,這項工作的可執行檔會使用該路徑存取現有的 Cloud Storage bucket。路徑開頭必須為 /mnt/disks/,後面接著您選擇的目錄或路徑。舉例來說,如要以名為 my-bucket 的目錄代表這個值區,請將掛接路徑設為 /mnt/disks/my-bucket

        3. 按一下 [完成]

      2. 在「工作數」欄位中,輸入這項工作的工作數。

        例如輸入 3

      3. 在「Parallelism」(平行處理) 欄位中,輸入要並行執行的工作數量。

        例如,輸入 1 (預設值)。

  4. 設定「其他設定」頁面:

    1. 按一下左側窗格中的「其他設定」。 「其他設定」頁面隨即開啟。

    2. 針對要掛接至這項工作的每個 Cloud Storage bucket,請執行下列操作:

      1. 在「儲存空間磁碟區」部分中,按一下「新增磁碟區」。 畫面上會顯示「New volume」(新增磁碟區) 視窗。

      2. 在「New volume」(新增卷宗) 視窗中,執行下列操作:

        1. 在「Volume type」(磁碟區類型) 區段中,選取「Cloud Storage bucket」(Cloud Storage bucket)

        2. 在「Storage Bucket name」(儲存空間 bucket 名稱) 欄位中,輸入現有 bucket 的名稱。

          例如,輸入您在這個作業的可執行檔中指定的值區。

        3. 在「掛接路徑」欄位中,輸入您在可執行檔中指定的 bucket 掛接路徑 (MOUNT_PATH)。

        4. 按一下 [完成]

  5. 選用:設定這項工作的其他欄位

  6. 選用:如要檢查工作設定,請在左側窗格中按一下「預覽」

  7. 點選「建立」

「Job details」(工作詳細資料) 頁面會顯示您建立的工作。

gcloud

如要使用 gcloud CLI 建立使用 Cloud Storage bucket 的工作,請使用 gcloud batch jobs submit 指令。在作業的 JSON 設定檔中,將值區掛接至 volumes 欄位。

舉例來說,如要建立將檔案輸出至 Cloud Storage 的工作,請執行下列指令:

  1. 建立含有下列內容的 JSON 檔案:

    {
        "taskGroups": [
            {
                "taskSpec": {
                    "runnables": [
                        {
                            "script": {
                                "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> MOUNT_PATH/output_task_${BATCH_TASK_INDEX}.txt"
                            }
                        }
                    ],
                    "volumes": [
                        {
                            "gcs": {
                                "remotePath": "BUCKET_PATH"
                            },
                            "mountPath": "MOUNT_PATH"
                        }
                    ]
                },
                "taskCount": 3
            }
        ],
        "logsPolicy": {
            "destination": "CLOUD_LOGGING"
        }
    }
    

    更改下列內容:

    • BUCKET_PATH:您要讓這項工作存取的值區目錄路徑,必須以值區名稱開頭。舉例來說,如果值區名稱為 BUCKET_NAME,路徑 BUCKET_NAME 代表值區的根目錄,路徑 BUCKET_NAME/subdirectory 代表 subdirectory 子目錄。
    • MOUNT_PATH:作業的可執行檔用來存取這個 bucket 的掛接路徑。路徑開頭必須為 /mnt/disks/,後面接著您選擇的目錄或路徑。舉例來說,如要以名為 my-bucket 的目錄代表這個值區,請將掛接路徑設為 /mnt/disks/my-bucket
  2. 執行下列指令:

    gcloud batch jobs submit JOB_NAME \
      --location LOCATION \
      --config JSON_CONFIGURATION_FILE
    

    更改下列內容:

    • JOB_NAME:作業名稱。
    • LOCATION:工作地點
    • JSON_CONFIGURATION_FILE:JSON 檔案的路徑,內含作業的設定詳細資料。

API

如要使用 Batch API 建立使用 Cloud Storage bucket 的工作,請使用 jobs.create 方法,並在 volumes 欄位中掛接 bucket。

POST https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/jobs?job_id=JOB_NAME

{
    "taskGroups": [
        {
            "taskSpec": {
                "runnables": [
                    {
                        "script": {
                            "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> MOUNT_PATH/output_task_${BATCH_TASK_INDEX}.txt"
                        }
                    }
                ],
                "volumes": [
                    {
                        "gcs": {
                            "remotePath": "BUCKET_PATH"
                        },
                        "mountPath": "MOUNT_PATH"
                    }
                ]
            },
            "taskCount": 3
        }
    ],
    "logsPolicy": {
            "destination": "CLOUD_LOGGING"
    }
}

更改下列內容:

  • PROJECT_ID:專案的專案 ID
  • LOCATION:工作地點
  • JOB_NAME:作業名稱。
  • BUCKET_PATH:您要讓這項工作存取的值區目錄路徑,必須以值區名稱開頭。舉例來說,如果值區名稱為 BUCKET_NAME,路徑 BUCKET_NAME 代表值區的根目錄,路徑 BUCKET_NAME/subdirectory 代表 subdirectory 子目錄。
  • MOUNT_PATH:作業的可執行檔用來存取這個 bucket 的掛接路徑。路徑開頭必須為 /mnt/disks/,後面接著您選擇的目錄或路徑。舉例來說,如要以名為 my-bucket 的目錄代表這個值區,請將掛接路徑設為 /mnt/disks/my-bucket

C++

C++

詳情請參閱 Batch C++ API 參考說明文件

如要向 Batch 進行驗證,請設定應用程式預設憑證。 詳情請參閱「為本機開發環境設定驗證」。

#include "google/cloud/batch/v1/batch_client.h"

  [](std::string const& project_id, std::string const& location_id,
     std::string const& job_id, std::string const& bucket_name) {
    // Initialize the request; start with the fields that depend on the sample
    // input.
    google::cloud::batch::v1::CreateJobRequest request;
    request.set_parent("projects/" + project_id + "/locations/" + location_id);
    request.set_job_id(job_id);
    // Most of the job description is fixed in this example; use a string to
    // initialize it, and then override the GCS remote path.
    auto constexpr kText = R"pb(
      task_groups {
        task_count: 4
        task_spec {
          compute_resource { cpu_milli: 500 memory_mib: 16 }
          max_retry_count: 2
          max_run_duration { seconds: 3600 }
          runnables {
            script {
              text: "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/share/output_task_${BATCH_TASK_INDEX}.txt"
            }
          }
          volumes { mount_path: "/mnt/share" }
        }
      }
      allocation_policy {
        instances {
          policy { machine_type: "e2-standard-4" provisioning_model: STANDARD }
        }
      }
      labels { key: "env" value: "testing" }
      labels { key: "type" value: "script" }
      logs_policy { destination: CLOUD_LOGGING }
    )pb";
    auto* job = request.mutable_job();
    if (!google::protobuf::TextFormat::ParseFromString(kText, job)) {
      throw std::runtime_error("Error parsing Job description");
    }
    job->mutable_task_groups(0)
        ->mutable_task_spec()
        ->mutable_volumes(0)
        ->mutable_gcs()
        ->set_remote_path(bucket_name);
    // Create a client and issue the request.
    auto client = google::cloud::batch_v1::BatchServiceClient(
        google::cloud::batch_v1::MakeBatchServiceConnection());
    auto response = client.CreateJob(request);
    if (!response) throw std::move(response).status();
    std::cout << "Job : " << response->DebugString() << "\n";
  }

Go

Go

詳情請參閱 Batch Go API 參考說明文件

如要向 Batch 進行驗證,請設定應用程式預設憑證。 詳情請參閱「為本機開發環境設定驗證」。

import (
	"context"
	"fmt"
	"io"

	batch "cloud.google.com/go/batch/apiv1"
	"cloud.google.com/go/batch/apiv1/batchpb"
	durationpb "google.golang.org/protobuf/types/known/durationpb"
)

// Creates and runs a job that executes the specified script
func createScriptJobWithBucket(w io.Writer, projectID, region, jobName, bucketName string) error {
	// projectID := "your_project_id"
	// region := "us-central1"
	// jobName := "some-job"
	// jobName := "some-bucket"

	ctx := context.Background()
	batchClient, err := batch.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("NewClient: %w", err)
	}
	defer batchClient.Close()

	// Define what will be done as part of the job.
	command := &batchpb.Runnable_Script_Text{
		Text: "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/share/output_task_${BATCH_TASK_INDEX}.txt",
	}

	// Specify the Google Cloud Storage bucket to mount
	volume := &batchpb.Volume{
		Source: &batchpb.Volume_Gcs{
			Gcs: &batchpb.GCS{
				RemotePath: bucketName,
			},
		},
		MountPath:    "/mnt/share",
		MountOptions: []string{},
	}

	// We can specify what resources are requested by each task.
	resources := &batchpb.ComputeResource{
		// CpuMilli is milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
		CpuMilli:  500,
		MemoryMib: 16,
	}

	taskSpec := &batchpb.TaskSpec{
		Runnables: []*batchpb.Runnable{{
			Executable: &batchpb.Runnable_Script_{
				Script: &batchpb.Runnable_Script{Command: command},
			},
		}},
		ComputeResource: resources,
		MaxRunDuration: &durationpb.Duration{
			Seconds: 3600,
		},
		MaxRetryCount: 2,
		Volumes:       []*batchpb.Volume{volume},
	}

	// Tasks are grouped inside a job using TaskGroups.
	taskGroups := []*batchpb.TaskGroup{
		{
			TaskCount: 4,
			TaskSpec:  taskSpec,
		},
	}

	// Policies are used to define on what kind of virtual machines the tasks will run on.
	// In this case, we tell the system to use "e2-standard-4" machine type.
	// Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
	allocationPolicy := &batchpb.AllocationPolicy{
		Instances: []*batchpb.AllocationPolicy_InstancePolicyOrTemplate{{
			PolicyTemplate: &batchpb.AllocationPolicy_InstancePolicyOrTemplate_Policy{
				Policy: &batchpb.AllocationPolicy_InstancePolicy{
					MachineType: "e2-standard-4",
				},
			},
		}},
	}

	// We use Cloud Logging as it's an out of the box available option
	logsPolicy := &batchpb.LogsPolicy{
		Destination: batchpb.LogsPolicy_CLOUD_LOGGING,
	}

	jobLabels := map[string]string{"env": "testing", "type": "script"}

	// The job's parent is the region in which the job will run
	parent := fmt.Sprintf("projects/%s/locations/%s", projectID, region)

	job := batchpb.Job{
		TaskGroups:       taskGroups,
		AllocationPolicy: allocationPolicy,
		Labels:           jobLabels,
		LogsPolicy:       logsPolicy,
	}

	req := &batchpb.CreateJobRequest{
		Parent: parent,
		JobId:  jobName,
		Job:    &job,
	}

	created_job, err := batchClient.CreateJob(ctx, req)
	if err != nil {
		return fmt.Errorf("unable to create job: %w", err)
	}

	fmt.Fprintf(w, "Job created: %v\n", created_job)

	return nil
}

Java

Java

詳情請參閱 Batch Java API 參考說明文件

如要向 Batch 進行驗證,請設定應用程式預設憑證。 詳情請參閱「為本機開發環境設定驗證」。

import com.google.cloud.batch.v1.AllocationPolicy;
import com.google.cloud.batch.v1.AllocationPolicy.InstancePolicy;
import com.google.cloud.batch.v1.AllocationPolicy.InstancePolicyOrTemplate;
import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.GCS;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.LogsPolicy.Destination;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.Runnable.Script;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.cloud.batch.v1.Volume;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreateWithMountedBucket {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";

    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "europe-central2";

    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "JOB_NAME";

    // Name of the bucket to be mounted for your Job.
    String bucketName = "BUCKET_NAME";

    createScriptJobWithBucket(projectId, region, jobName, bucketName);
  }

  // This method shows how to create a sample Batch Job that will run
  // a simple command on Cloud Compute instances.
  public static void createScriptJobWithBucket(String projectId, String region, String jobName,
      String bucketName)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the `batchServiceClient.close()` method on the client to safely
    // clean up any remaining background resources.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setScript(
                  Script.newBuilder()
                      .setText(
                          "echo Hello world from task ${BATCH_TASK_INDEX}. >> "
                              + "/mnt/share/output_task_${BATCH_TASK_INDEX}.txt")
                      // You can also run a script from a file. Just remember, that needs to be a
                      // script that's already on the VM that will be running the job.
                      // Using setText() and setPath() is mutually exclusive.
                      // .setPath("/tmp/test.sh")
                      .build())
              .build();

      Volume volume = Volume.newBuilder()
          .setGcs(GCS.newBuilder()
              .setRemotePath(bucketName)
              .build())
          .setMountPath("/mnt/share")
          .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(500)
              // In MiB.
              .setMemoryMib(16)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .addVolumes(volume)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(4).setTaskSpec(task).build();

      // Policies are used to define on what kind of virtual machines the tasks will run on.
      // In this case, we tell the system to use "e2-standard-4" machine type.
      // Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
      InstancePolicy instancePolicy =
          InstancePolicy.newBuilder().setMachineType("e2-standard-4").build();

      AllocationPolicy allocationPolicy =
          AllocationPolicy.newBuilder()
              .addInstances(InstancePolicyOrTemplate.newBuilder().setPolicy(instancePolicy).build())
              .build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              .setAllocationPolicy(allocationPolicy)
              .putLabels("env", "testing")
              .putLabels("type", "script")
              .putLabels("mount", "bucket")
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(
                  LogsPolicy.newBuilder().setDestination(Destination.CLOUD_LOGGING).build())
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());
    }
  }
}

Node.js

Node.js

詳情請參閱 Batch Node.js API 參考說明文件

如要向 Batch 進行驗證,請設定應用程式預設憑證。 詳情請參閱「為本機開發環境設定驗證」。

/**
 * TODO(developer): Uncomment and replace these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
/**
 * The region you want to the job to run in. The regions that support Batch are listed here:
 * https://cloud.google.com/batch/docs/get-started#locations
 */
// const region = 'us-central-1';
/**
 * The name of the job that will be created.
 * It needs to be unique for each project and region pair.
 */
// const jobName = 'YOUR_JOB_NAME';
/**
 * The name of the bucket to be mounted.
 */
// const bucketName = 'YOUR_BUCKET_NAME';

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

// Define what will be done as part of the job.
const task = new batch.TaskSpec();
const runnable = new batch.Runnable();
runnable.script = new batch.Runnable.Script();
runnable.script.text =
  'echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/share/output_task_${BATCH_TASK_INDEX}.txt';
// You can also run a script from a file. Just remember, that needs to be a script that's
// already on the VM that will be running the job. Using runnable.script.text and runnable.script.path is mutually
// exclusive.
// runnable.script.path = '/tmp/test.sh'
task.runnables = [runnable];

const gcsBucket = new batch.GCS();
gcsBucket.remotePath = bucketName;
const gcsVolume = new batch.Volume();
gcsVolume.gcs = gcsBucket;
gcsVolume.mountPath = '/mnt/share';
task.volumes = [gcsVolume];

// We can specify what resources are requested by each task.
const resources = new batch.ComputeResource();
resources.cpuMilli = 2000; // in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
resources.memoryMib = 16;
task.computeResource = resources;

task.maxRetryCount = 2;
task.maxRunDuration = {seconds: 3600};

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup();
group.taskCount = 4;
group.taskSpec = task;

// Policies are used to define on what kind of virtual machines the tasks will run on.
// In this case, we tell the system to use "e2-standard-4" machine type.
// Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
const allocationPolicy = new batch.AllocationPolicy();
const policy = new batch.AllocationPolicy.InstancePolicy();
policy.machineType = 'e2-standard-4';
const instances = new batch.AllocationPolicy.InstancePolicyOrTemplate();
instances.policy = policy;
allocationPolicy.instances = [instances];

const job = new batch.Job();
job.name = jobName;
job.taskGroups = [group];
job.allocationPolicy = allocationPolicy;
job.labels = {env: 'testing', type: 'script'};
// We use Cloud Logging as it's an option available out of the box
job.logsPolicy = new batch.LogsPolicy();
job.logsPolicy.destination = batch.LogsPolicy.Destination.CLOUD_LOGGING;

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateJob() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const response = await batchClient.createJob(request);
  console.log(response);
}

await callCreateJob();

Python

Python

詳情請參閱 Batch Python API 參考說明文件

如要向 Batch 進行驗證,請設定應用程式預設憑證。 詳情請參閱「為本機開發環境設定驗證」。

from google.cloud import batch_v1


def create_script_job_with_bucket(
    project_id: str, region: str, job_name: str, bucket_name: str
) -> batch_v1.Job:
    """
    This method shows how to create a sample Batch Job that will run
    a simple command on Cloud Compute instances.

    Args:
        project_id: project ID or project number of the Cloud project you want to use.
        region: name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
        job_name: the name of the job that will be created.
            It needs to be unique for each project and region pair.
        bucket_name: name of the bucket to be mounted for your Job.

    Returns:
        A job object representing the job created.
    """
    client = batch_v1.BatchServiceClient()

    # Define what will be done as part of the job.
    task = batch_v1.TaskSpec()
    runnable = batch_v1.Runnable()
    runnable.script = batch_v1.Runnable.Script()
    runnable.script.text = "echo Hello world from task ${BATCH_TASK_INDEX}. >> /mnt/share/output_task_${BATCH_TASK_INDEX}.txt"
    task.runnables = [runnable]

    gcs_bucket = batch_v1.GCS()
    gcs_bucket.remote_path = bucket_name
    gcs_volume = batch_v1.Volume()
    gcs_volume.gcs = gcs_bucket
    gcs_volume.mount_path = "/mnt/share"
    task.volumes = [gcs_volume]

    # We can specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 500  # in milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
    resources.memory_mib = 16
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Tasks are grouped inside a job using TaskGroups.
    # Currently, it's possible to have only one task group.
    group = batch_v1.TaskGroup()
    group.task_count = 4
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    allocation_policy = batch_v1.AllocationPolicy()
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy.instances = [instances]

    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy
    job.labels = {"env": "testing", "type": "script", "mount": "bucket"}
    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

使用網路檔案系統

您可以使用現有的網路檔案系統 (NFS) (例如 Filestore 檔案共用區),透過 Google Cloud 控制台、gcloud CLI 或 Batch API 建立工作。

建立使用 NFS 的作業前,請確認網路的防火牆已正確設定,允許作業的 VM 與 NFS 之間的流量。詳情請參閱設定 Filestore 的防火牆規則

以下範例說明如何建立工作,指定並掛接 NFS。這項工作也有 3 項工作,每項工作都會執行指令碼,在名為 output_task_TASK_INDEX.txt 的 NFS 中建立檔案,其中 TASK_INDEX 是每項工作的索引:012

控制台

如要使用 Google Cloud 控制台建立使用 NFS 的作業,請按照下列步驟操作:

  1. 前往 Google Cloud 控制台的「Job list」(工作清單) 頁面。

    前往工作清單

  2. 按一下「 Create」(建立)。「Create batch job」(建立批次工作) 頁面隨即開啟。左側窗格會選取「工作詳細資料」頁面。

  3. 設定「工作詳細資料」頁面:

    1. 選用:在「Job name」(工作名稱) 欄位中,自訂工作名稱。

      例如輸入 example-nfs-job

    2. 設定「工作詳細資料」部分:

      1. 在「新增可執行項目」視窗中,為這項工作新增至少一個指令碼或容器。

        舉例來說,請執行下列操作:

        1. 勾選「指令碼」核取方塊。畫面上會顯示文字方塊。

        2. 在文字方塊中輸入下列指令碼:

          echo Hello world from task ${BATCH_TASK_INDEX}. >> MOUNT_PATH/output_task_${BATCH_TASK_INDEX}.txt
          

          MOUNT_PATH 替換為作業可執行檔用來存取這個 NFS 的掛接路徑。路徑開頭必須為 /mnt/disks/,後面接著您選擇的目錄或路徑。舉例來說,如要以名為 my-nfs 的目錄表示這個 NFS,請將掛接路徑設為 /mnt/disks/my-nfs

        3. 按一下 [完成]

      2. 在「工作數」欄位中,輸入這項工作的工作數。

        例如輸入 3

      3. 在「Parallelism」(平行處理) 欄位中,輸入要並行執行的工作數量。

        例如,輸入 1 (預設值)。

  4. 設定「其他設定」頁面:

    1. 按一下左側窗格中的「其他設定」。 「其他設定」頁面隨即開啟。

    2. 針對要掛接至這項工作的每個 Cloud Storage bucket,請執行下列操作:

      1. 在「儲存空間磁碟區」部分中,按一下「新增磁碟區」。 畫面上會顯示「New volume」(新增磁碟區) 視窗。

      2. 在「New volume」(新增卷宗) 視窗中,執行下列操作:

        1. 在「Volume type」(磁碟區類型) 區段中,選取「Network file system」(網路檔案系統)

        2. 在「檔案伺服器」欄位中,輸入伺服器的 IP 位址,該伺服器是您在這個工作可執行檔中指定的 NFS 所在位置。

          舉例來說,如果 NFS 是 Filestore 檔案共用區,請指定 Filestore 執行個體的 IP 位址,您可以說明 Filestore 執行個體來取得該位址。

        3. 在「Remote path」(遠端路徑) 欄位中,輸入可存取上一步驟中指定 NFS 的路徑。

          NFS 目錄路徑的開頭必須為 /, 後方加上 NFS 的根目錄。

        4. 在「Mount path」(掛接路徑) 欄位中,輸入您在上一個步驟中指定的 NFS 掛接路徑 (MOUNT_PATH)。

    3. 按一下 [完成]

  5. 選用:設定這項工作的其他欄位

  6. 選用:如要檢查工作設定,請在左側窗格中按一下「預覽」

  7. 點選「建立」

「Job details」(工作詳細資料) 頁面會顯示您建立的工作。

gcloud

如要使用 gcloud CLI 建立使用 NFS 的工作,請使用 gcloud batch jobs submit 指令。在工作的 JSON 設定檔中,於 volumes 欄位掛接 NFS。

  1. 建立含有下列內容的 JSON 檔案:

    {
        "taskGroups": [
            {
                "taskSpec": {
                    "runnables": [
                        {
                            "script": {
                                "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> MOUNT_PATH/output_task_${BATCH_TASK_INDEX}.txt"
                            }
                        }
                    ],
                    "volumes": [
                        {
                            "nfs": {
                                "server": "NFS_IP_ADDRESS",
                                "remotePath": "NFS_PATH"
                            },
                            "mountPath": "MOUNT_PATH"
                        }
                    ]
                },
                "taskCount": 3
            }
        ],
        "logsPolicy": {
            "destination": "CLOUD_LOGGING"
        }
    }
    

    更改下列內容:

    • NFS_IP_ADDRESS:NFS 的 IP 位址。舉例來說,如果 NFS 是 Filestore 檔案共用區,請指定 Filestore 執行個體的 IP 位址,您可以說明 Filestore 執行個體來取得該位址。
    • NFS_PATH:這項工作要存取的 NFS 目錄路徑,必須以 / 開頭,後面接著 NFS 的根目錄。舉例來說,如果 Filestore 檔案共用區名為 FILE_SHARE_NAME,則路徑 /FILE_SHARE_NAME 代表檔案共用區的根目錄,路徑 /FILE_SHARE_NAME/subdirectory 代表 subdirectory 子目錄。
    • MOUNT_PATH:作業的可執行檔用來存取這個 NFS 的掛接路徑。路徑開頭必須為 /mnt/disks/,後面接著您選擇的目錄或路徑。舉例來說,如要以名為 my-nfs 的目錄表示這個 NFS,請將掛接路徑設為 /mnt/disks/my-nfs
  2. 執行下列指令:

    gcloud batch jobs submit JOB_NAME \
      --location LOCATION \
      --config JSON_CONFIGURATION_FILE
    

    更改下列內容:

    • JOB_NAME:作業名稱。
    • LOCATION:工作地點
    • JSON_CONFIGURATION_FILE:JSON 檔案的路徑,內含作業的設定詳細資料。

API

如要使用 Batch API 建立使用 NFS 的工作,請使用 jobs.create 方法,並在 volumes 欄位中掛接 NFS。

POST https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/jobs?job_id=JOB_NAME

   {
    "taskGroups": [
        {
            "taskSpec": {
                "runnables": [
                    {
                        "script": {
                            "text": "echo Hello world from task ${BATCH_TASK_INDEX}. >> MOUNT_PATH/output_task_${BATCH_TASK_INDEX}.txt"
                        }
                    }
                ],
                "volumes": [
                    {
                        "nfs": {
                            "server": "NFS_IP_ADDRESS",
                            "remotePath": "NFS_PATH"
                        },
                        "mountPath": "MOUNT_PATH"
                    }
                ]
            },
            "taskCount": 3
        }
    ],
    "logsPolicy": {
        "destination": "CLOUD_LOGGING"
    }
}

更改下列內容:

  • PROJECT_ID:專案的專案 ID
  • LOCATION:工作地點
  • JOB_NAME:作業名稱。
  • NFS_IP_ADDRESS:網路檔案系統的 IP 位址。舉例來說,如果 NFS 是 Filestore 檔案共用區,請指定 Filestore 執行個體的 IP 位址,您可以說明 Filestore 執行個體來取得該位址。
  • NFS_PATH:這項工作要存取的 NFS 目錄路徑,必須以 / 開頭,後面接著 NFS 的根目錄。舉例來說,如果 Filestore 檔案共用區名為 FILE_SHARE_NAME,則路徑 /FILE_SHARE_NAME 代表檔案共用區的根目錄,路徑 /FILE_SHARE_NAME/subdirectory 代表子目錄。
  • MOUNT_PATH:作業的可執行檔用來存取這個 NFS 的掛接路徑。路徑開頭必須為 /mnt/disks/,後面接著您選擇的目錄或路徑。舉例來說,如要以名為 my-nfs 的目錄表示這個 NFS,請將掛接路徑設為 /mnt/disks/my-nfs

Java


import com.google.cloud.batch.v1.AllocationPolicy;
import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.NFS;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.cloud.batch.v1.Volume;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreateScriptJobWithNfs {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";

    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "europe-central2";

    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "JOB_NAME";

    // The path of the NFS directory that you want this job to access.
    String nfsPath = "NFS_PATH";
    // The IP address of the Network File System.
    String nfsIpAddress = "NFS_IP_ADDRESS";

    createScriptJobWithNfs(projectId, region, jobName, nfsPath, nfsIpAddress);
  }

  // This method shows how to create a batch script job that specifies and mounts a NFS.
  public static Job createScriptJobWithNfs(String projectId, String region, String jobName,
                                            String nfsPath, String nfsIpAddress)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setScript(
                  Runnable.Script.newBuilder()
                      .setText(
                          "echo Hello world from task ${BATCH_TASK_INDEX}. >> "
                              + "/mnt/share/output_task_${BATCH_TASK_INDEX}.txt")
                      // You can also run a script from a file. Just remember, that needs to be a
                      // script that's already on the VM that will be running the job.
                      // Using setText() and setPath() is mutually exclusive.
                      // .setPath("/tmp/test.sh")
                      .build())
              .build();

      // Describes a volume and parameters for it to be mounted to a VM.
      Volume volume = Volume.newBuilder()
          .setNfs(NFS.newBuilder()
              .setServer(nfsIpAddress)
              .setRemotePath(nfsPath)
              .build())
          .setMountPath("/mnt/share")
          .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(500)
              // In MiB.
              .setMemoryMib(16)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .addVolumes(volume)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(4).setTaskSpec(task).build();

      // Policies are used to define on what kind of virtual machines the tasks will run on.
      // In this case, we tell the system to use "e2-standard-4" machine type.
      // Read more about machine types here:
      // https://cloud.google.com/compute/docs/machine-types
      AllocationPolicy.InstancePolicy instancePolicy =
          AllocationPolicy.InstancePolicy.newBuilder().setMachineType("e2-standard-4").build();

      AllocationPolicy allocationPolicy =
          AllocationPolicy.newBuilder()
              .addInstances(AllocationPolicy.InstancePolicyOrTemplate.newBuilder()
                      .setPolicy(instancePolicy).build())
              .build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              .setAllocationPolicy(allocationPolicy)
              .putLabels("env", "testing")
              .putLabels("type", "script")
              .putLabels("mount", "bucket")
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(LogsPolicy.newBuilder()
                      .setDestination(LogsPolicy.Destination.CLOUD_LOGGING).build())
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }
}

Node.js

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'europe-central2';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'batch-nfs-job';
// The path of the NFS directory that you want this job to access.
const nfsPath = '/your_nfs_path';
// The IP address of the Network File System.
const nfsIpAddress = '0.0.0.0';
// The mount path that the job's tasks use to access the NFS.
const mountPath = '/mnt/disks';

// Define what will be done as part of the job.
const runnable = new batch.Runnable({
  script: new batch.Runnable.Script({
    commands: [
      '-c',
      'echo Hello world from task ${BATCH_TASK_INDEX}. >> ' +
        '/mnt/share/output_task_${BATCH_TASK_INDEX}.txt',
    ],
  }),
});

// Define a volume that uses NFS.
const volume = new batch.Volume({
  nfs: new batch.NFS({
    server: nfsIpAddress,
    remotePath: nfsPath,
  }),
  mountPath,
});

// Specify what resources are requested by each task.
const computeResource = new batch.ComputeResource({
  // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
  cpuMilli: 500,
  // In MiB.
  memoryMib: 16,
});

const task = new batch.TaskSpec({
  runnables: [runnable],
  volumes: [volume],
  computeResource,
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

// Policies are used to define on what kind of virtual machines the tasks will run on.
// In this case, we tell the system to use "e2-standard-4" machine type.
// Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
const instancePolicy = new batch.AllocationPolicy.InstancePolicy({
  machineType: 'e2-standard-4',
});

const allocationPolicy = new batch.AllocationPolicy.InstancePolicyOrTemplate({
  instances: [{policy: instancePolicy}],
});

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  allocationPolicy,
  labels: {env: 'testing', type: 'script'},
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchNfsJob() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchNfsJob();

Python

from google.cloud import batch_v1


def create_job_with_network_file_system(
    project_id: str,
    region: str,
    job_name: str,
    mount_path: str,
    nfs_ip_address: str,
    nfs_path: str,
) -> batch_v1.Job:
    """
    Creates a Batch job with status events that mounts a Network File System (NFS).
    Function mounts an NFS volume using the provided NFS server, IP address and path.

    Args:
        project_id (str): project ID or project number of the Cloud project you want to use.
        region (str): name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/locations
        job_name (str): the name of the job that will be created.
            It needs to be unique for each project and region pair.
        mount_path (str): The mount path that the job's tasks use to access the NFS.
        nfs_ip_address (str): The IP address of the NFS server (e.g., Filestore instance).
            Documentation on how to create a
            Filestore instance is available here: https://cloud.google.com/filestore/docs/create-instance-gcloud
        nfs_path (str): The path of the NFS directory that the job accesses.
            The path must start with a / followed by the root directory of the NFS.

    Returns:
        batch_v1.Job: The created Batch job object containing configuration details.
    """
    client = batch_v1.BatchServiceClient()

    # Create a runnable with a script that writes a message to a file
    runnable = batch_v1.Runnable()
    runnable.script = batch_v1.Runnable.Script()
    runnable.script.text = f"echo Hello world from task ${{BATCH_TASK_INDEX}}. >> {mount_path}/output_task_${{BATCH_TASK_INDEX}}.txt"

    # Define a volume that uses NFS
    volume = batch_v1.Volume()
    volume.nfs = batch_v1.NFS(server=nfs_ip_address, remote_path=nfs_path)
    volume.mount_path = mount_path

    # Create a task specification and assign the runnable and volume to it
    task = batch_v1.TaskSpec()
    task.runnables = [runnable]
    task.volumes = [volume]

    # Specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 2000  # in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
    resources.memory_mib = 16  # in MiB
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Create a task group and assign the task specification to it
    group = batch_v1.TaskGroup()
    group.task_count = 1
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    # Create the job and assign the task group and allocation policy to it
    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy
    job.labels = {"env": "testing", "type": "container"}
    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    # Create the job request and set the job and job ID
    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

後續步驟