CI/CD pipeline for developing and delivering containerized apps

This document describes an integrated set of Google Cloud tools to set up a system for development, for continuous integration (CI), and for continuous delivery (CD) that you can use to develop and deploy applications to Google Kubernetes Engine (GKE). This reference architecture document is intended for both software developers and operators. It assumes that you're familiar with running gcloud commands on Google Cloud and with deploying application containers to GKE.

Architecture

The following diagram shows the resources that are used in this architecture:

This architecture includes the following components:

1. Cloud Code as a development workspace. As part of this workspace, you can see changes in the development cluster, which runs on minikube. You run Cloud Code and the minikube cluster in Cloud Shell. Cloud Shell is an online development environment accessible from your browser. It has compute resources, memory, an integrated development environment, (IDE), and it also has Cloud Code installed.

2. Cloud Build to build and test the application—the "CI" part of the pipeline

   This part of the pipeline includes the following actions:

   • Cloud Build monitors changes to the source repository, using a Cloud Build trigger.
   • When a change is committed into the main branch, the Cloud Build trigger does the following:
     • Rebuilds the application container.
     • Places build artifacts in a Cloud Storage bucket.
     • Places the application container in Artifact Registry.
     • Runs tests on the container.
     • Calls Cloud Deploy to deploy the container to the staging environment. In this example, the staging environment is a Google Kubernetes Engine cluster.
   • If the build and tests are successful, you can then use Cloud Deploy to promote the container from staging to production.

3. Cloud Deploy to manage the deployment—the "CD" part of the pipeline. In this part of the pipeline, Cloud Deploy does the following:

   • Registers a delivery pipeline and targets. The targets represent the staging and production clusters.
   • Creates a Cloud Storage bucket and stores the Skaffold rendering source and rendered manifests in that bucket.
   • Generates a new release for each source code change.
   • Deploys the application to the production environment. For this deployment to production, an operator (or other designated person) manually approves the deployment. In this architecture, the production environment is a Google Kubernetes Engine cluster.

In this architecture, configuration is shared among the development, staging, and production environments through Skaffold, a command-line tool that facilitates continuous development for Kubernetes-native applications.

Google Cloud stores the application's source code in GitHub.

This architecture uses Google Cloud products for most of the components of the system, with Skaffold enabling the integration of the system. Because Skaffold is open source, you can use these principles to create a similar system using a combination of Google Cloud, in-house, and third-party components. The modularity of this solution means that you can adopt it incrementally as part of your development and deployment pipeline.

Use cases

The following are the key features of this integrated system:

• Develop and deploy faster.

  The development loop is efficient because you can validate changes in the developer workspace. Deployment is fast because the automated CI/CD system and increased parity across the environments allow you to detect more issues when you deploy changes to production.

• Benefit from increased parity across development, staging, and production.

  The components of this system use a common set of Google Cloud tools.

• Reuse configurations across the different environments.

  This reuse is done with Skaffold, which allows a common configuration format for the different environments. It also allows developers and operators to update and use the same configuration.

• Apply governance early in the workflow.

  This system applies validation tests for governance at production and in the CI system and development environment. Applying governance in the development environment allows problems to be found and fixed earlier.

• Let opinionated tooling manage your software delivery.

  Continuous delivery is fully managed, separating the stages of your CD pipeline from the details of rendering and deploying.

Deployment

To deploy this architecture, see Develop and deploy containerized apps using a CI/CD pipeline.

What's next

• To learn how to deploy into a private GKE instance, see Deploying to a private cluster on a Virtual Private Cloud network.
• For information about how to implement, improve, and measure deployment automation, see Deployment automation.