Set parallelism for jobs

You use parallelism to specify the maximum number of tasks in a job execution that can run in parallel. By default, tasks will be started as quickly as possible, up to a maximum that varies depending on how many CPUs you are using.

Lowering parallelism limits how many tasks run in parallel. This is useful in cases where one of your backing resources, such as a database, has limited scaling and cannot handle a large number of parallel requests.

Required roles

To get the permissions that you need to configure Cloud Run jobs, ask your administrator to grant you the following IAM roles:

For a list of IAM roles and permissions that are associated with Cloud Run, see Cloud Run IAM roles and Cloud Run IAM permissions. If your Cloud Run job interfaces with Google Cloud APIs, such as Cloud Client Libraries, see the service identity configuration guide. For more information about granting roles, see deployment permissions and manage access.

Configure parallelism

To specify parallelism for a Cloud Run job:

Console

  1. In the Google Cloud console, go to the Cloud Run jobs page:

    Go to Cloud Run

  2. Select Jobs from the menu, and click Deploy container to fill out the initial job settings page. If you are configuring an existing job, select the job, then click Edit.

  3. Click Container(s), Volumes, Connections, Security to expand the job properties page.

  4. Click the General tab.

    image

    • Select Run as many tasks concurrently as possible for best performance. If you need to lower the number of concurrent tasks, for example, if your backing resources are limited, select Limit the number of concurrent tasks and then specify an integer between 0 and the maximum number of tasks running in parallel allowed by quota that don't exceed the number of tasks.

  5. Click Create or Update.

gcloud

  1. For a job you are creating:

    gcloud run jobs create JOB_NAME --image IMAGE_URL --parallelism PARALLELISM

    Replace

    • JOB_NAME with the name of your job.
    • IMAGE_URL: a reference to the container image—for example, us-docker.pkg.dev/cloudrun/container/job:latest.
    • PARALLELISM with an integer between 0 and the maximum number of tasks running in parallel, allowed by quota, that don't exceed the number of tasks.

  2. For a job you are updating:

    gcloud run jobs update JOB_NAME --parallelism PARALLELISM

YAML

  1. If you are creating a new job, skip this step. If you are updating an existing job, download its YAML configuration:

    gcloud run jobs describe JOB_NAME --format export > job.yaml

  2. Update the parallelism: attribute:

    apiVersion: run.googleapis.com/v1
kind: Job
metadata:
  name: JOB_NAME
spec:
  template:
    spec:
      parallelism: PARALLELISM
      template:
        spec:
          containers:
          - image: IMAGE

    Replace PARALLELISM. Specify an integer between 0 and the maximum number of tasks running in parallel allowed by quota that don't exceed the number of tasks.

    You can also specify more configuration such as environment variables or memory limits.

  3. Update the existing job configuration:

    gcloud run jobs replace job.yaml

Terraform

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.

Add the following to a google_cloud_run_v2_job resource in your Terraform configuration:

resource "google_cloud_run_v2_job" "default" {
  name     = "cloud-run-job-parallelism"
  location = "us-central1"

  deletion_protection = false # set to "true" in production

  template {
    task_count  = 3
    parallelism = 3

    template {
      containers {
        image = "us-docker.pkg.dev/cloudrun/container/job:latest"
      }
    }
  }
}

View parallelism settings

To view the current parallelism settings for your Cloud Run job:

Console

  1. In the Google Cloud console, go to the Cloud Run jobs page:

    Go to Cloud Run jobs

  2. Click the job you are interested in to open the Job details page.

  3. Click View and Edit job configuration.

  4. Locate the parallelism setting in the configuration details.

gcloud

  1. Use the following command:

    gcloud run jobs describe JOB_NAME

  2. Locate the parallelism setting in the returned configuration.