Deploy worker pools to Cloud Run

This page shows how to deploy container images to a new Cloud Run worker pool or to a new revision of an existing Cloud Run worker pool.

Worker pools are a Cloud Run resource specifically designed for non-request workloads. In contrast to Cloud Run services, worker pools do not have the following features:

  • Public endpoint/URL
  • Requirement to expose an ingress port
  • Request-driven autoscaling

For an example walkthrough of deploying a new worker pool, see Deploy a sample worker pool quickstart.

Required roles

To get the permissions that you need to deploy Cloud Run worker pools, ask your administrator to grant you the following IAM roles:

  • Cloud Run Developer (roles/run.developer) on the Cloud Run worker pool
  • Service Account User (roles/iam.serviceAccountUser) on the identity that your worker pools uses to interact with other Google Cloud services
  • Artifact Registry Reader (roles/artifactregistry.reader) on the Artifact Registry repository of the deployed container image

For a list of IAM roles and permissions that are associated with Cloud Run, see Cloud Run IAM roles and Cloud Run IAM permissions. If your Cloud Run worker pool interfaces with Google Cloud APIs, such as Cloud Client Libraries, see the service identity configuration guide. For more information about granting roles, see deployment permissions and manage access.

Supported container registries and images

You can directly use container images stored in Artifact Registry, or Docker Hub. Google recommends the use of Artifact Registry. Docker Hub images are cached for up to one hour.

You can use container images from other public or private registries (like JFrog Artifactory, Nexus, or GitHub Container Registry), by setting up an Artifact Registry remote repository.

You should only consider Docker Hub for deploying popular container images such as Docker Official Images or Docker Sponsored OSS images. For higher availability, Google recommends deploying these Docker Hub images using an Artifact Registry remote repository.

Cloud Run does not support container image layers larger than 9.9 GB when deploying from Docker Hub or an Artifact Registry remote repository with an external registry.

Deploy worker pools

You can deploy worker pools in the following ways:

Deploy a new worker pool

You can specify a container image with a tag (for example, us-docker.pkg.dev/my-project/container/my-image:latest) or with an exact digest (for example, us-docker.pkg.dev/my-project/container/my-image@sha256:41f34ab970ee...).

Deploying a worker pool for the first time creates its first revision. Note that revisions are immutable. If you deploy from a container image tag, it will be resolved to a digest and the revision will always serve this particular digest.

Follow the instructions, using the Google Cloud CLI.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To deploy a worker pool container image:

    1. Run the following command: 

      gcloud beta run worker-pools deploy WORKER_POOL --image IMAGE_URL

      Replace:

      • WORKER_POOL with the name of the worker pool you want to deploy to. If the worker pool does not exist yet, this command creates the worker pool during the deployment. You can omit this parameter entirely, but you will be prompted for the worker pool name if you omit it. Worker pool names must be 49 characters or fewer, use a unique name per region and project, and must not share the same name as an existing service name from your project.
      • IMAGE_URL with a reference to the container image that contains the worker pool, such as us-docker.pkg.dev/cloudrun/container/worker-pool:latest. Note that if you don't supply the --image flag, the deploy command attempts to deploy from source code.

    2. Wait for the deployment to finish. Upon successful completion, Cloud Run displays a success message along with the revision information about the deployed worker pool.

      To deploy to a different location from the one you set using the run/region gcloud properties, use:

      gcloud beta run worker-pools deploy WORKER_POOL --region REGION

Deploy a new revision of an existing worker pool

Note that changing configuration settings for a worker pool results in the creation of a new revision, even if there is no change to the container image. Each revision created is immutable.

The container image is imported by Cloud Run when deployed. Cloud Run keeps this copy of the container image as long as it is used by a revision.

Follow these instructions, using the Google Cloud CLI:

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To deploy a container image:

    1. Run the command: 

      gcloud beta run worker-pools deploy WORKER_POOL --image IMAGE_URL

      Replace:

      • WORKER_POOL with the name of the worker pool you want to deploy to. If the worker pool does not exist yet, this command creates the worker pool during the deployment. You can omit this parameter entirely, but you will be prompted for the worker pool name if you omit it. Worker pool names must be 49 characters or fewer, use a unique name per region and project, and must not share the same name as an existing service name from your project.
      • IMAGE_URL with a reference to the container image that contains the worker pool, such as us-docker.pkg.dev/cloudrun/container/worker-pool:latest. Note that if you don't supply the --image flag, the deploy command attempts to deploy from source code.

      The revision suffix is assigned automatically for new revisions. If you want to supply your own revision suffix, use the gcloud CLI parameter --revision-suffix.

    2. Wait for the deployment to finish. Upon successful completion, Cloud Run displays a success message along with the revision information about the deployed worker pool.

Deploy images from other Google Cloud projects

To deploy images from other Google Cloud projects, you or your administrator must grant the required IAM roles to the deployer account and the Cloud Run service agent.

For the required roles for the deployer account, see required roles.

To grant the Cloud Run service agent the required roles, see the following instructions:

  1. In the Google Cloud console, open the project for your Cloud Run worker pool.

    Go to the IAM page

  2. Select Include Google-provided role grants.

  3. Copy the email of the Cloud Run service agent. It has the suffix @serverless-robot-prod.iam.gserviceaccount.com

  4. Open the project that owns the container registry you want to use.

    Go to the IAM page

  5. Click Add to add a new principal.

  6. In the New principals field, paste in the email of the service account that you copied earlier.

  7. In the Select a role drop-down menu, if you are using Container Registry, select the role Storage -> Storage Object Viewer. If you are using Artifact Registry, select the role Artifact Registry -> Artifact Registry Reader.

  8. Deploy the container image to the project that contains your Cloud Run worker pool.

Deploying images from other registries

To deploy public or private container images that are not stored in Artifact Registry or Docker Hub, set up an Artifact Registry remote repository.

Artifact Registry remote repositories allow you to:

  • Deploy any public container image, for example, GitHub Container Registry (ghcr.io).
  • Deploy container images from private repositories that require authentication, for example, JFrog Artifactory or Nexus.

If using an Artifact Registry remote repository is not an option, you can temporarily pull and push container images to Artifact Registry by deploying them to Cloud Run using docker push. Cloud Run imports the container image during deployment, and afterwards, you can delete the image from Artifact Registry.

Deploy multiple containers (sidecars) to a worker pool

In a Cloud Run deployment with sidecars, there is one main worker pool container and one or more sidecar containers. The sidecars can communicate with each other and with the worker pool container using a localhost port. The localhost port varies depending on the containers you are using.

You can deploy up to 10 containers per instance including the worker pool container. All containers within an instance share the same network namespace and can also share files using an in-memory shared volume.

You can require all deployments to use a specific sidecar by creating custom organization policies.

Deploy a service with sidecar containers

Follow these instructions, using the Google Cloud CLI to deploy multiple containers to a Cloud Run worker pool:

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To deploy multiple containers to a worker pool, run the following command:

    gcloud beta run worker-pools deploy WORKER_POOL \
 --container WORKER_POOL_CONTAINER_NAME \
 --image='WORKER_POOL_IMAGE' \
 --container SIDECAR_CONTAINER_NAME \
 --image='SIDECAR_IMAGE'

    Replace:

    • WORKER_POOL with the name of the worker pool you are deploying to. If you omit this parameter, you will be prompted for the worker pool name.
    • WORKER_POOL_CONTAINER_NAME with a name for the worker pool container.
    • IMAGE_URL with a reference to the container image that contains the worker pool, such as us-docker.pkg.dev/cloudrun/container/worker-pool:latest.
    • SIDECAR_CONTAINER_NAME with a name for the sidecar container, for example sidecar.
    • SIDECAR_IMAGE with a reference to the sidecar container image.

    To configure each container in the deploy command, supply each container's configuration after the container parameters, for example:

    gcloud beta run worker-pools deploy WORKER_POOL \
  --container CONTAINER_1_NAME \
  --image='WORKER_POOL_IMAGE' \
  --set-env-vars=KEY=VALUE \
  --container SIDECAR_CONTAINER_NAME \
  --image='SIDECAR_IMAGE' \
  --set-env-vars=KEY_N=VALUE_N

  3. Wait for the deployment to finish. Upon successful completion, Cloud Run displays a success message.

Notable features available to deployments with sidecars

You can specify the container start up order within a deployment with multiple containers, if you have dependencies that require some containers to start up before other containers in the deployment.

If you have containers that depend on other containers, you must use healthchecks in your deployment. When you use healthchecks, Cloud Run follows the container startup order, verifying the health of each container before starting the next container. Without healthchecks, Cloud Run tries to start all containers, even if the containers they depend on are not running yet or failed to start.

Multiple containers within a single instance can access a shared in-memory volume, which is accessible to each container using mount points that you create.

What's next

After you deploy a new worker pool, you can do the following: