Reporting Policy Controller audit violations in Security Command Center

Last reviewed 2023-04-17 UTC

This tutorial shows platform security administrators how to view and manage policy violations for Kubernetes resources alongside other vulnerability and security findings in Security Command Center. In this tutorial, you can use either Policy Controller or Open Policy Agent (OPA) Gatekeeper.

Architecture

Policy Controller checks, audits, and enforces your Kubernetes cluster resources' compliance with policies related to security, regulations, or business rules. Policy Controller is built from the OPA Gatekeeper open source project.

The audit functionality in Policy Controller and OPA Gatekeeper lets you implement detective controls that periodically evaluate resources against policies. If an issue is detected, the controls create violations for resources that don't conform to the policies. These violations are stored in the cluster, and you can query them using Kubernetes tools such as kubectl.

To make these violations visible and to help you take actions, you can use Security Command Center. Security Command Center provides a dashboard and APIs for surfacing, understanding, and remediating security and data risks across an organization for Google Cloud resources, Kubernetes resources, and hybrid or multi-cloud resources.

Security Command Center displays possible security risks and policy violations, called findings. Findings come from sources, which are mechanisms that can detect and report risks and violations. Security Command Center includes built-in services, and you can add third-party sources and your own sources.

This tutorial and associated source code shows you how to create a source and findings in Security Command Center for Policy Controller and OPA Gatekeeper policy violations.

The following diagram shows the architecture that is implemented in this tutorial:

Architecture with a source, controller, and sync.

As the preceding diagram shows, in this tutorial you create a source in Security Command Center using a command-line tool. You deploy a controller to a Google Kubernetes Engine (GKE) cluster to synchronize Policy Controller and OPA Gatekeeper constraint violations to findings in Security Command Center.

If you want to see how to synchronize policy violations for Google Cloud resources, try out our tutorial about how to create policy-compliant Google Cloud resources using Config Connector and Policy Controller.

Objectives

  • Create a policy and a resource that violates the policy.
  • Create a source in Security Command Center.
  • Create a finding in Security Command Center from an OPA Gatekeeper policy violation using a command-line tool.
  • Deploy a controller to the GKE cluster to periodically synchronize findings in Security Command Center from OPA Gatekeeper policy violations.
  • View findings in your terminal and in the Google Cloud console.

Costs

In this document, you use the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish the tasks that are described in this document, you can avoid continued billing by deleting the resources that you created. For more information, see Clean up.

Before you begin

  1. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  2. Make sure that billing is enabled for your Google Cloud project.

  3. To complete this tutorial, you must have an appropriate editor role for Security Command Center at the organization level, such as Security Center Admin Editor. Your organization administrator can grant you this role.
  4. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

Preparing the environment

  1. In Cloud Shell, set the Google Cloud project that you want to use for this tutorial:

    gcloud config set project PROJECT_ID
    

    Replace PROJECT_ID with your Google Cloud project ID. When you run this command, Cloud Shell creates an exported environment variable called GOOGLE_CLOUD_PROJECT that contains your project ID.

  2. Enable the Resource Manager, GKE, Security Command Center, and Service Usage APIs:

    gcloud services enable \
        cloudresourcemanager.googleapis.com \
        container.googleapis.com \
        securitycenter.googleapis.com \
        serviceusage.googleapis.com
    

Creating a GKE cluster

  1. In Cloud Shell, create a GKE cluster with Workload Identity enabled:

    gcloud container clusters create gatekeeper-securitycenter-tutorial \
        --enable-ip-alias \
        --release-channel regular \
        --workload-pool $GOOGLE_CLOUD_PROJECT.svc.id.goog \
        --zone us-central1-f
    

    This command creates the cluster in the us-central1-f zone. You can use a different zone or region.

  2. Grant yourself the cluster-admin cluster role:

    kubectl create clusterrolebinding cluster-admin-binding \
        --clusterrole cluster-admin \
        --user $(gcloud config get-value core/account)
    

    You need this role later to create some of the Kubernetes resources used by the controller. You also need it if you install the open source OPA Gatekeeper distribution.

Installing the policy tool

If you have a managed GKE cluster, follow the instructions to install Policy Controller, otherwise install the OPA Gatekeeper distribution.

Policy Controller

Install Policy Controller by following the installation instructions.

Use an audit interval of 60 seconds.

OPA Gatekeeper

  1. In Cloud Shell, define the OPA Gatekeeper version that you want to install:

    GATEKEEPER_VERSION=v3.10.0
    
  2. Install OPA Gatekeeper:

    kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/$GATEKEEPER_VERSION/deploy/gatekeeper.yaml
    
  3. Verify that OPA Gatekeeper is installed:

    kubectl rollout status deploy gatekeeper-controller-manager \
        -n gatekeeper-system
    

    When the installation completes, the output displays deployment "gatekeeper-controller-manager" successfully rolled out.

Creating a policy

A policy in Policy Controller and OPA Gatekeeper consists of a constraint template and a constraint. The constraint template contains the policy logic. The constraint specifies where the policy applies and specifies input parameters for the policy logic.

In this section, you create a policy for Kubernetes Pods and a Pod that violates the policy.

  1. In Cloud Shell, clone the OPA Gatekeeper library repository, go to the repository directory, and check out a known commit:

    git clone https://github.com/open-policy-agent/gatekeeper-library.git \
        ~/gatekeeper-library
    
    cd ~/gatekeeper-library
    
    git checkout 1da0facae99658accb73c291cb79f497fcddf641
    
  2. Create a Pod called nginx-disallowed in the default namespace:

    kubectl apply -f library/general/allowedrepos/samples/repo-must-be-openpolicyagent/example_disallowed.yaml
    

    The following is the manifest that you apply to create the Pod:

    apiVersion: v1
    kind: Pod
    metadata:
      name: nginx-disallowed
    spec:
      containers:
        - name: nginx
          image: nginx
          resources:
            limits:
              cpu: "100m"
              memory: "30Mi"
    

    This Pod uses a container image from a repository that isn't approved by the policy.

  3. Create a constraint template called k8sallowedrepos:

    kubectl apply -f library/general/allowedrepos/template.yaml
    

    The following is the constraint template manifest:

    apiVersion: templates.gatekeeper.sh/v1beta1
    kind: ConstraintTemplate
    metadata:
      name: k8sallowedrepos
      annotations:
        description: >- 
          Requires container images to begin with a string from the specified list.
    spec:
      crd:
        spec:
          names:
            kind: K8sAllowedRepos
          validation:
            # Schema for the `parameters` field
            openAPIV3Schema:
              type: object
              properties:
                repos:
                  description: The list of prefixes a container image is allowed to have.
                  type: array
                  items:
                    type: string
      targets:
        - target: admission.k8s.gatekeeper.sh
          rego: |
            package k8sallowedrepos
    
            violation[{"msg": msg}] {
              container := input.review.object.spec.containers[_]
              satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
              not any(satisfied)
              msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
            }
    
            violation[{"msg": msg}] {
              container := input.review.object.spec.initContainers[_]
              satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
              not any(satisfied)
              msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
            }
    
  4. Create a constraint called repo-is-openpolicyagent:

    kubectl apply -f library/general/allowedrepos/samples/repo-must-be-openpolicyagent/constraint.yaml
    

    The following is the constraint manifest:

    apiVersion: constraints.gatekeeper.sh/v1beta1
    kind: K8sAllowedRepos
    metadata:
      name: repo-is-openpolicyagent
    spec:
      match:
        kinds:
          - apiGroups: [""]
            kinds: ["Pod"]
        namespaces:
          - "default"
      parameters:
        repos:
          - "openpolicyagent/"
    

Auditing constraints

The audit controller in Policy Controller and OPA Gatekeeper periodically evaluates resources against constraints. This auditing lets you detect policy-violating resources that were created before you created the constraint.

  1. In Cloud Shell, view violations for all constraints by querying using the constraint category:

    kubectl get constraint -o json | jq '.items[].status.violations'
    

    The output is the following:

    [
      {
        "enforcementAction": "deny",
        "kind": "Pod",
        "message": "container <nginx> has an invalid image repo <nginx>, allowed repos are [\"openpolicyagent\"]",
        "name": "nginx-disallowed",
        "namespace": "default"
      }
    ]
    

    There is a violation for the Pod that you created before you created the constraint. If you see null instead of the preceding output, the Policy Controller or OPA Gatekeeper audit hasn't run since you created the constraint. By default, the audit runs every minute. Wait a minute and try again.

Creating a Security Command Center source

Security Command Center records findings against sources. Follow these steps to create a source for findings from Policy Controller and OPA Gatekeeper:

  1. In Cloud Shell, create a Google service account and store the service account name in an environment variable:

    SOURCES_ADMIN_SA=$(gcloud iam service-accounts create \
        securitycenter-sources-admin \
        --display-name "Security Command Center sources admin" \
        --format 'value(email)')
    

    You use this Google service account to administer Security Command Center sources.

  2. Define an environment variable that contains your Google Cloud organization ID:

    ORGANIZATION_ID=$(gcloud projects get-ancestors $GOOGLE_CLOUD_PROJECT \
        --format json | jq -r '.[] | select (.type=="organization") | .id')
    
  3. Grant the Security Center Sources Admin role to the sources admin Google service account at the organization level:

    gcloud organizations add-iam-policy-binding $ORGANIZATION_ID \
        --member "serviceAccount:$SOURCES_ADMIN_SA" \
        --role roles/securitycenter.sourcesAdmin
    

    This role provides the securitycenter.sources.* permissions that are required to administer sources.

  4. Grant the Service Usage Consumer role to the sources admin Google service account at the organization level:

    gcloud organizations add-iam-policy-binding $ORGANIZATION_ID \
        --member "serviceAccount:$SOURCES_ADMIN_SA" \
        --role roles/serviceusage.serviceUsageConsumer
    

    This role provides the serviceusage.services.use permission to use projects in the organization for quota and billing purposes.

  5. Grant yourself the Service Account Token Creator role for the sources admin Google service account:

    gcloud iam service-accounts add-iam-policy-binding \
        $SOURCES_ADMIN_SA \
        --member "user:$(gcloud config get-value account)" \
        --role roles/iam.serviceAccountTokenCreator
    

    This role allows your user identity to impersonate, or act as, the Google service account.

  6. Download the latest version of the gatekeeper-securitycenter command-line tool for your platform and make it executable:

    VERSION=v0.4.0
    
    curl -Lo gatekeeper-securitycenter "https://github.com/GoogleCloudPlatform/gatekeeper-securitycenter/releases/download/${VERSION}/gatekeeper-securitycenter_$(uname -s)_$(uname -m)"
    
    chmod +x gatekeeper-securitycenter
    
  7. Use the gatekeeper-securitycenter tool to create a Security Command Center source for your organization. Capture the full source name in an environment variable.

    export SOURCE_NAME=$(./gatekeeper-securitycenter sources create \
        --organization $ORGANIZATION_ID \
        --display-name "Gatekeeper" \
        --description "Reports violations from Policy Controller audits" \
        --impersonate-service-account $SOURCES_ADMIN_SA | jq -r '.name')
    

    This command creates a source with the display name Gatekeeper. This display name is visible in Security Command Center. You can use a different display name and description.

    If you get a response with the error message, The caller does not have permission, wait a minute, and then try again. This error can happen if the Identity and Access Management (IAM) bindings haven't taken effect yet.

Creating findings using the command line

You can create Security Command Center findings from Policy Controller and OPA Gatekeeper constraint violations using the