Creating a CI/CD pipeline with Azure Pipelines and Cloud Run

Connecting Azure Pipelines to Artifact Registry

Before you can set up continuous integration for the CloudDemo app, you must connect Azure Pipelines to Artifact Registry. This connection allows Azure Pipelines to publish container images to Artifact Registry.

Set up a service account for publishing images

Create a Google Cloud service account in your production project:

1. In the Google Cloud console, switch to the production project.

2. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

3. Initialize the following environment variables:

   DEV_PROJECT_ID=DEV_PROJECT_ID
   PROD_PROJECT_ID=PROD_PROJECT_ID
   

   Replace the following:

   • DEV_PROJECT_ID: the project ID of your development project
   • PROD_PROJECT_ID: the project ID of your production project

4. Enable the Artifact Registry API in the production project:

   gcloud services enable artifactregistry.googleapis.com \
       --project=$PROD_PROJECT_ID
   

5. Create an Artifact Registry repository to store Docker your images:

   gcloud artifacts repositories create docker  \
       --project=$PROD_PROJECT_ID \
       --repository-format=docker \
       --location REGION
   

   Replace REGION with the region of your Artifact Registry repository, for example us-central1.

6. Create a service account that Azure Pipelines uses to publish Docker images:

   gcloud iam service-accounts create azure-pipelines-publisher \
       --display-name="Azure Pipelines Publisher" \
       --project=$PROD_PROJECT_ID
   

7. Grant the Artifact Registry Writer role (roles/artifactregistry.writer) to the service account to allow Azure Pipelines to push to Artifact Registry:

   AZURE_PIPELINES_PUBLISHER=azure-pipelines-publisher@$PROD_PROJECT_ID.iam.gserviceaccount.com
   
   gcloud projects add-iam-policy-binding $PROD_PROJECT_ID \
       --member serviceAccount:$AZURE_PIPELINES_PUBLISHER \
       --role roles/artifactregistry.writer \
       --project=$PROD_PROJECT_ID
   

8. Generate a service account key:

   gcloud iam service-accounts keys create azure-pipelines-publisher.json \
       --iam-account $AZURE_PIPELINES_PUBLISHER \
       --project=$PROD_PROJECT_ID
   
   tr -d '\n' < azure-pipelines-publisher.json > azure-pipelines-publisher-oneline.json
   

9. View the content of the service account key file:

   echo $(<azure-pipelines-publisher-oneline.json)
   

   You need the service account key in one of the following steps.

Create a service connection for Artifact Registry

In Azure Pipelines, create a new service connection for Artifact Registry:

1. In the Azure DevOps menu, select Project settings, and then select Pipelines > Service connections.
2. Click Create service connection.
3. From the list, select Docker Registry, and then click Next.

4. In the dialog, enter values for the following fields:

   • Registry type: Others

   • Docker Registry: https://REGION-docker.pkg.dev/PROD_PROJECT_ID/docker/

     Replace the following:

     • REGION: the region of your Artifact Registry repository
     • PROD_PROJECT_ID: the name of your production project

   • Docker ID: _json_key

   • Password: Paste the content of azure-pipelines-publisher-oneline.json.

   • Service connection name: gcr-tutorial

5. Click Save to create the connection.

Building continuously

You can now use Azure Pipelines to set up continuous integration. For each commit that's pushed to the Git repository, Azure Pipelines builds the code and packages the build artifacts into a Docker container. The container is then published to Artifact Registry.

The repository already contains the following Dockerfile:

#
# Copyright 2020 Google LLC
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
# 
#   http://www.apache.org/licenses/LICENSE-2.0
# 
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

FROM mcr.microsoft.com/dotnet/aspnet:6.0
EXPOSE 8080

#------------------------------------------------------------------------------
# Copy publishing artifacts.
#------------------------------------------------------------------------------

WORKDIR /app
COPY CloudDemo.MvcCore/bin/Release/net6.0/publish/ /app/

ENV ASPNETCORE_URLS=http://0.0.0.0:8080

#------------------------------------------------------------------------------
# Run application in Kestrel.
#------------------------------------------------------------------------------

ENTRYPOINT ["dotnet", "CloudDemo.MvcCore.dll"]

You now create a new pipeline that uses YAML syntax:

1. Using Visual Studio or a command-line git client, clone your new Git repository.
2. In the root of the repository, create a file named azure-pipelines.yml.

3. Copy the following code into the file:

   resources:
   - repo: self
     fetchDepth: 1
   pool:
     vmImage: ubuntu-22.04
   trigger:
   - master
   variables:
     TargetFramework: 'net6.0'
     BuildConfiguration: 'Release'
     DockerImageName: 'PROD_PROJECT_ID/docker/CloudDemo'
   steps:
   - task: DotNetCoreCLI@2
     displayName: Publish
     inputs:
       projects: 'applications/clouddemo/netcore/CloudDemo.MvcCore.sln'
       publishWebProjects: false
       command: publish
       arguments: '--configuration $(BuildConfiguration) --framework=$(TargetFramework)'
       zipAfterPublish: false
       modifyOutputPath: false
   - task: PublishBuildArtifacts@1
     displayName: 'Publish Artifact'
     inputs:
       PathtoPublish: '$(build.artifactstagingdirectory)'
   - task: Docker@2
     displayName: 'Login to Container Registry'
     inputs:
       command: login
       containerRegistry: 'gcr-tutorial'
   - task: Docker@2
     displayName: 'Build and push image'
     inputs:
       Dockerfile: 'applications/clouddemo/netcore/Dockerfile'
       command: buildAndPush
       repository: '$(DockerImageName)'
   

   Replace PROJECT_ID with the name of your production project, and then save the file.

   Because Cloud Run is a Linux-based environment, the pipeline uses Linux-based build agents.

4. Commit your changes and push them to Azure Pipelines.

   Visual Studio

   1. Open Team Explorer and click the Home icon.
   2. Click Changes.
   3. Enter a commit message like Add pipeline definition.
   4. Click Commit All and Push.

   Command line

   1. Stage all modified files:

      git add -A
      

   2. Commit the changes to the local repository:

      git commit -m "Add pipeline definition"
      

   3. Push the changes to Azure DevOps:

      git push
      

5. In the Azure DevOps menu, select Pipelines and then click Create Pipeline.

6. Select Azure Repos Git.

7. Select your repository.

8. On the Review your pipeline YAML page, click Run.

   A new build is triggered. It might take about two minutes for the build to complete.

9. To verify that the image has been published to Artifact Registry, switch to your production project in the Google Cloud console, select Artifact Registry > docker, and then click CloudDemo.

   A single image and the tag of this image are displayed. The tag corresponds to the numeric ID of the build that was run in Azure Pipelines.

Deploying continuously

With Azure Pipelines automatically building your code and publishing Docker images for each commit, you can now turn your attention to deployment.

Unlike some other continuous integration systems, Azure Pipelines distinguishes between building and deploying, and it provides a specialized set of tools labeled Release Management for all deployment-related tasks.

Azure Pipelines Release Management is built around these concepts:

• A release refers to a set of artifacts that make up a specific version of your app and that are usually the result of a build process.
• Deployment refers to the process of taking a release and deploying it into a specific environment.
• A deployment performs a set of tasks, which can be grouped in jobs.
• Stages let you segment your pipeline and can be used to orchestrate deployments to multiple environments—for example development and testing environments.

You set up your release pipeline to be triggered whenever a new build is completed. The pipeline consists of two stages: development and production. In each stage, the release pipeline uses the Docker image that the build pipeline produces, and then the pipeline deploys it to Cloud Run.

The build pipeline that you configured previously tags each Docker image with the build ID before publishing it to Artifact Registry. Therefore, in the release pipeline, you use the $BUILD_BUILDID variable to identify the right Docker image to deploy.

Configure Cloud Run

Cloud Run is a fully managed, serverless environment, so you don't need to provision any infrastructure. To help keep your Cloud Run deployments secure, you need to set up IAM.

Deploying and running a Cloud Run service involves multiple identities, as the following diagram shows.

Each of these identities is implemented as a service account and is used for a specific purpose, as the following table describes.

You created and configured the Azure Pipelines Publisher service account. In the following sections, you create and configure the remaining service accounts.

Configure the Cloud Run service account

1. Open Cloud Shell.

2. Initialize the following environment variables:

   DEV_PROJECT_ID=DEV_PROJECT_ID
   PROD_PROJECT_ID=PROD_PROJECT_ID
   

   Replace the following:

   • DEV_PROJECT_ID: the project ID of your development project
   • PROD_PROJECT_ID: the project ID of your production project

3. Enable the Cloud Run and Compute Engine APIs in both the development and product projects:

   gcloud services enable run.googleapis.com --project=$DEV_PROJECT_ID
   gcloud services enable run.googleapis.com --project=$PROD_PROJECT_ID
   

   Enabling these APIs causes the Cloud Run service agent accounts to be created in your projects.

4. Grant the two Cloud Run service agent accounts access to Artifact Registry in the production project where your Docker images are stored:

   DEV_PROJECT_NUMBER=$(gcloud projects describe $DEV_PROJECT_ID \
       --format='value(projectNumber)')
   PROD_PROJECT_NUMBER=$(gcloud projects describe $PROD_PROJECT_ID \
       --format='value(projectNumber)')
   
   gcloud projects add-iam-policy-binding $PROD_PROJECT_ID \
       --member=serviceAccount:service-$DEV_PROJECT_NUMBER@serverless-robot-prod.iam.gserviceaccount.com \
       --role roles/artifactregistry.reader
   
   gcloud projects add-iam-policy-binding $PROD_PROJECT_ID \
       --member=serviceAccount:service-$PROD_PROJECT_NUMBER@serverless-robot-prod.iam.gserviceaccount.com \
       --role roles/artifactregistry.reader
   

Configure the CloudDemo runner account

You can now configure the CloudDemo runner account, which is a custom runtime service account for the CloudDemo service:

• Create a service account named CloudDemo-runner:

  gcloud iam service-accounts create clouddemo-runner \
      --display-name="CloudDemo Runner" \
      --project=$DEV_PROJECT_ID
  
  gcloud iam service-accounts create clouddemo-runner \
      --display-name="CloudDemo Runner" \
      --project=$PROD_PROJECT_ID
  
  DEV_CLOUDDEMO_RUNNER=clouddemo-runner@$DEV_PROJECT_ID.iam.gserviceaccount.com
  
  PROD_CLOUDDEMO_RUNNER=clouddemo-runner@$PROD_PROJECT_ID.iam.gserviceaccount.com
  

Configure the Azure Pipelines Deployer account

Finally, create and configure the Azure Pipelines Deployer account, which the Azure release pipeline uses to deploy to Cloud Run.

1. Create a service account named azure-pipelines-deployer:

   gcloud iam service-accounts create azure-pipelines-deployer \
       --display-name="Azure Pipelines Deployer" \
       --project=$PROD_PROJECT_ID
   
   AZURE_PIPELINES_DEPLOYER=azure-pipelines-deployer@$PROD_PROJECT_ID.iam.gserviceaccount.com
   

2. Assign the required IAM roles to deploy new Cloud Run services or revisions in the development project:

   gcloud projects add-iam-policy-binding $DEV_PROJECT_ID \
       --member serviceAccount:$AZURE_PIPELINES_DEPLOYER \
       --role roles/run.admin
   
   gcloud iam service-accounts add-iam-policy-binding \
       $DEV_CLOUDDEMO_RUNNER \
       --member=serviceAccount:$AZURE_PIPELINES_DEPLOYER \
       --role="roles/iam.serviceAccountUser" \
       --project=$DEV_PROJECT_ID
   

3. Assign the same set of roles for the production project:

   gcloud projects add-iam-policy-binding $PROD_PROJECT_ID \
       --member serviceAccount:$AZURE_PIPELINES_DEPLOYER \
       --role roles/run.admin
   
   gcloud iam service-accounts add-iam-policy-binding \
       $PROD_CLOUDDEMO_RUNNER \
       --member=serviceAccount:$AZURE_PIPELINES_DEPLOYER \
       --role="roles/iam.serviceAccountUser" \
       --project=$PROD_PROJECT_ID
   

4. Generate a service account key:

   gcloud iam service-accounts keys create azure-pipelines-deployer.json \
       --iam-account=$AZURE_PIPELINES_DEPLOYER \
       --project=$PROD_PROJECT_ID
   
   cat azure-pipelines-deployer.json | base64 -w 0
   

   You use the output of this command when you configure the release pipeline.

Configure the release pipeline

You can now return to Azure Pipelines to automate the deployment, which includes the following steps:

• Deploying to the development environment.
• Requesting manual approval before initiating a deployment to the production environment.
• Deploying to the production environment.

Create a release definition

First, create a release definition:

1. In the Azure DevOps menu, select Pipelines > Releases.
2. Click New pipeline.
3. From the list of templates, select Empty job.
4. When you're prompted for a name for the stage, enter Development.
5. At the top of the screen, name the pipeline CloudDemo.
6. In the pipeline diagram, next to Artifacts, click Add.
7. Select Build, and then add the following settings:
   • Source type: Build
   • Source (build pipeline): Select the build definition (there should be only one option)
   • Default version: Latest
   • Source Alias: build
8. Click Add.
9. On the Artifact box, click Continuous deployment trigger (the lightning bolt icon) to add a deployment trigger.
10. Under Continuous deployment trigger, set the switch to Enabled.
11. Click Save.

12. Enter a comment if you want, and then confirm by clicking OK.

    The pipeline displays similar to the following.

    

Deploy to the development project

With the release definition created, you can now configure the Cloud Run deployment to the development project.

1. In the menu, switch to the Tasks tab.
2. Click Agent job.
3. Set Agent specification to ubuntu-22.04.
4. Next to Agent job, click Add a task to agent job add to add a step to the phase.
5. Select the Command line task and click Add.

6. Click the newly added task and configure the following settings:

   1. Display name: Deploy image to development project

   2. Script:

      gcloud auth activate-service-account \
          --quiet \
          --key-file <(echo $(ServiceAccountKey) | base64 -d) && \
      gcloud run deploy clouddemo \
          --quiet \
          --service-account=clouddemo-runner@$(CloudRun.ProjectId.Development).iam.gserviceaccount.com \
          --allow-unauthenticated \
          --image=$(CloudRun.Region)-docker.pkg.dev/$(ContainerRegistry.ProjectId)/docker/clouddemo:$BUILD_BUILDID \
          --platform=managed \
          --region=$(CloudRun.Region) \
          --project=$(CloudRun.ProjectId.Development)
      

      This command obtains a service account key from an environment variable and then uses the gcloud CLI to deploy the application to Cloud Run. The gcloud CLI is available by default on Azure Pipelines agents.

7. Switch to the Variables tab and add the following variables.

   Name	Value	Secret
   ServiceAccountKey	Service account key created for azure-pipelines-deployer earlier.	Yes
   ContainerRegistry.ProjectId	Project ID of your production project.
   CloudRun.Region	Region you selected earlier to deploy Artifact Registry resources in.
   CloudRun.ProjectId.Development	Project ID of your development project.
   CloudRun.ProjectId.Production	Project ID of your production project.

8. Click Save.

9. Enter a comment if you want, and then confirm by clicking OK.

Deploy to the production cluster

Finally, you configure the deployment to the production project:

1. In the menu, switch to the Pipeline tab.
2. In the Stages box, select Add > New stage.
3. From the list of templates, select Empty job.
4. When you're prompted for a name for the stage, enter Production.
5. Click the lightning bolt icon of the newly created stage.

6. Configure the following settings:

   1. Select trigger: After stage
   2. Stages: Development
   3. Pre-deployment approvals: (enabled)
   4. Approvers: Select your username.

   The pipeline displays a view similar to the following.

   

7. Switch to the Tasks tab.

8. Hold the mouse over the Tasks tab and select Tasks > Production.

9. Click Agent job.

10. Set Agent specification to ubuntu-22.04.

11. Click Add a task to agent job add to add a step to the phase.

12. Select the Command line task, and then click Add.

13. Click the newly added task and configure the following settings:

    1. Display name: Deploy image to production project

    2. Script:

       gcloud auth activate-service-account \
           --quiet \
           --key-file <(echo $(ServiceAccountKey) | base64 -d) && \
       gcloud run deploy clouddemo \
           --quiet \
           --service-account=clouddemo-runner@$(CloudRun.ProjectId.Production).iam.gserviceaccount.com \
           --allow-unauthenticated \
           --image=$(CloudRun.Region)-docker.pkg.dev/$(ContainerRegistry.ProjectId)/docker/clouddemo:$BUILD_BUILDID \
           --platform=managed \
           --region=$(CloudRun.Region) \
           --project=$(CloudRun.ProjectId.Production)
       

14. Click Save.

15. Enter a comment if you want, and then confirm by clicking OK.

Run the pipeline

Now that you've configured the entire pipeline, you can test it by performing a source code change:

1. On your local computer, open the file applications\clouddemo\netcore\CloudDemo.MvcCore\Views\Home\Index.cshtml from the Git repository you cloned earlier.
2. In line 26, change the value of ViewBag.Title from Home Page to Home Page Cloud Run.

3. Commit your changes, and then push them to Azure Pipelines.

   Visual Studio

   1. Open Team Explorer and click the Home icon.
   2. Click Changes.
   3. Enter a commit message like Change site title.
   4. Click Commit All and Push.

   Command line

   1. Stage all modified files:

      git add -A
      

   2. Commit the changes to the local repository:

      git commit -m "Change site title"
      

   3. Push the changes to Azure Pipelines:

      git push
      

4. In the Azure DevOps menu, select Pipelines. A build is triggered.

5. After the build is finished, select Pipelines > Releases. A release process is initiated.

6. Click Release-1 to open the details page, and wait for the status of the Development stage to switch to Succeeded.

7. In the Google Cloud console, switch to the development project.

8. In the menu, select Compute > Cloud Run.

   The clouddemo service is deployed successfully.

   

9. Click clouddemo to view more details.

   A URL is displayed which shows that Cloud Run provisioned the service.

10. Open the URL in a new browser tab to verify that the CloudDemo app is deployed and uses the custom title.

11. In Azure Pipelines, click Approve (adjacent to the Production stage) to promote the deployment to the production environment.

12. (Optional) Enter a comment.

13. Confirm by clicking Approve, and wait for the status of the Production environment to switch to Succeeded.

14. In the Google Cloud console, switch to the production project.

15. In the menu, select Compute > Cloud Run.

    The clouddemo service is deployed to the production project.

16. Click clouddemo to view more details.

    A URL displays which shows that Cloud Run provisioned the service.

17. Open the URL in a new browser tab to verify that the CloudDemo app is deployed to production and that it uses the custom title.