Use the trusted directory check

This page shows you how to use the Binary Authorization continuous validation (CV) trusted directory check to check that images associated with Pods running on CV-enabled Google Kubernetes Engine (GKE) clusters were deployed from trusted directories.

Costs

This guide uses the following Google Cloud services:

  • Binary Authorization, but CV is available free of charge during the Preview stage
  • GKE

To generate a cost estimate based on your projected usage, use the pricing calculator.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. Install the Google Cloud CLI.

  3. To initialize the gcloud CLI, run the following command: 

    gcloud init

  4. Create or select a Google Cloud project.

    • Create a Google Cloud project:

      gcloud projects create PROJECT_ID

      Replace PROJECT_ID with a name for the Google Cloud project you are creating.

    • Select the Google Cloud project that you created:

      gcloud config set project PROJECT_ID

      Replace PROJECT_ID with your Google Cloud project name.

  5. Make sure that billing is enabled for your Google Cloud project.

  6. Enable the Artifact Registry, Binary Authorization, Google Kubernetes Engine APIs: 

    gcloud services enable artifactregistry.googleapis.com binaryauthorization.googleapis.com container.googleapis.com
  7. Install the Google Cloud CLI.

  8. To initialize the gcloud CLI, run the following command: 

    gcloud init

  9. Create or select a Google Cloud project.

    • Create a Google Cloud project:

      gcloud projects create PROJECT_ID

      Replace PROJECT_ID with a name for the Google Cloud project you are creating.

    • Select the Google Cloud project that you created:

      gcloud config set project PROJECT_ID

      Replace PROJECT_ID with your Google Cloud project name.

  10. Make sure that billing is enabled for your Google Cloud project.

  11. Enable the Artifact Registry, Binary Authorization, Google Kubernetes Engine APIs: 

    gcloud services enable artifactregistry.googleapis.com binaryauthorization.googleapis.com container.googleapis.com
  12. Ensure that the gcloud CLI is updated to the latest version.
  13. Install the kubectl command-line tool.
  14. If your Binary Authorization policies and GKE clusters are in different projects, make sure that Binary Authorization is enabled in both projects.

Required roles

This section shows you how to set roles for this check.

Overview

If you run all of the products that are mentioned in this guide in the same project, you don't need to set any permissions. Binary Authorization configures the roles correctly when you enable it. If you run the products in different projects, you must set role as described in this section.

To ensure that the cluster project's Binary Authorization service agent has the necessary permissions to evaluate the CV trusted directory check, ask your administrator to grant the cluster project's Binary Authorization service agent the following IAM roles on the project:

  • If your cluster project is different from the policy project: Binary Authorization Policy Evaluator (roles/binaryauthorization.policyEvaluator) - the cluster project Binary Authorization Service Agent, for it to access the policy project

For more information about granting roles, see Manage access to projects, folders, and organizations.

Your administrator might also be able to give the cluster project's Binary Authorization service agent the required permissions through custom roles or other predefined roles.

Grant roles using the gcloud CLI

If the project where you run your cluster is different from the project where the policy resides, do the following to grant permission for the cluster project's Binary Authorization service agent to access the policy in the policy project.

  1. Get the cluster project's Binary Authorization service agent:

    PROJECT_NUMBER=$(gcloud projects list --filter="projectId:CLUSTER_PROJECT_ID" \
  --format="value(PROJECT_NUMBER)")
CLUSTER_SERVICE_ACCOUNT="service-$PROJECT_NUMBER@gcp-sa-binaryauthorization.iam.gserviceaccount.com"

    Replace CLUSTER_PROJECT_ID with the project ID of the cluster.

  2. Allow CV to evaluate the policy on the cluster:

    gcloud projects add-iam-policy-binding POLICY_PROJECT_ID \
    --member="serviceAccount:$CLUSTER_SERVICE_ACCOUNT" \
    --role='roles/binaryauthorization.policyEvaluator'

    Replace POLICY_PROJECT_ID with the ID of the project that contains your policy.

Create a platform policy

To create a CV platform policy with a trusted directory check, do the following:

  1. Create the trusted directory policy YAML file:

    cat > /tmp/my-policy.yaml <<EOF
gkePolicy:
  checkSets:
    checks:
      trustedDirectoryCheck:
        trustedDirPatterns:
        - PATTERN1
        - PATTERN2
      displayName: CHECK_DISPLAY_NAME
    displayName: CHECK_SET_DISPLAY_NAME
EOF

    Replace the following:

    • PATTERN1: a list item with a directory pattern
    • PATTERN2: a list item with a directory pattern
    • CHECK_DISPLAY_NAME: an optional display name for the trusted directory check
    • CHECK_SET_DISPLAY_NAME: an optional display name for the trusted directory check

    The following are examples of patterns:

    • asia-east1-docker.pkg.dev/my-project/my-repo: trusts only this repository
    • europe-central1-docker.pkg.dev/my-project/my-repo/test*: trusts only this repository and repositories immediately under it that begin with test
    • us-central1-docker.pkg.dev/my-project/my-repo/**: trusts only this repository and all repos under it

  2. Create the platform policy:

    Before using any of the command data below, make the following replacements:

    • POLICY_ID: A platform policy ID of your choice. If the policy is in another project, you can use the full resource name: projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID.
    • POLICY_PATH: A path to the policy file.
    • POLICY_PROJECT_ID: The policy project ID.

    Execute the following command:

    Linux, macOS, or Cloud Shell

    gcloud beta container binauthz policy create POLICY_ID \
    --platform=gke \
    --policy-file=POLICY_PATH \
    --project=POLICY_PROJECT_ID

    Windows (PowerShell)

    gcloud beta container binauthz policy create POLICY_ID `
    --platform=gke `
    --policy-file=POLICY_PATH `
    --project=POLICY_PROJECT_ID

    Windows (cmd.exe)

    gcloud beta container binauthz policy create POLICY_ID ^
    --platform=gke ^
    --policy-file=POLICY_PATH ^
    --project=POLICY_PROJECT_ID

Enable CV

You can create a new cluster or update an existing cluster to use CV monitoring with check-based platform policies.

Create a cluster that uses CV monitoring

In this section, you create a cluster that uses only CV monitoring with check-based platform policies.

Before using any of the command data below, make the following replacements:

  • CLUSTER_NAME: a cluster name.
  • LOCATION: the location—for example, us-central1 or asia-south1.
  • POLICY_PROJECT_ID: the ID of the project where the policy is stored.
  • POLICY_ID: the policy ID.
  • CLUSTER_PROJECT_ID: the cluster project ID.

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud beta container clusters create CLUSTER_NAME \
    --location=LOCATION \
    --binauthz-evaluation-mode=POLICY_BINDINGS \
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID \
    --project=CLUSTER_PROJECT_ID

Windows (PowerShell)

gcloud beta container clusters create CLUSTER_NAME `
    --location=LOCATION `
    --binauthz-evaluation-mode=POLICY_BINDINGS `
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID `
    --project=CLUSTER_PROJECT_ID

Windows (cmd.exe)

gcloud beta container clusters create CLUSTER_NAME ^
    --location=LOCATION ^
    --binauthz-evaluation-mode=POLICY_BINDINGS ^
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID ^
    --project=CLUSTER_PROJECT_ID

Create cluster that uses enforcement and CV monitoring

In this section, you create a cluster that uses both project-singleton policy enforcement and CV monitoring with check-based platform policies:

Before using any of the command data below, make the following replacements:

  • CLUSTER_NAME: a cluster name.
  • LOCATION: the location—for example, us-central1 or asia-south1.
  • POLICY_PROJECT_ID: the ID of the project where the policy is stored.
  • POLICY_ID: the policy ID.
  • CLUSTER_PROJECT_ID: the cluster project ID.

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud beta container clusters create CLUSTER_NAME \
    --location=LOCATION \
    --binauthz-evaluation-mode=POLICY_BINDINGS_AND_PROJECT_SINGLETON_POLICY_ENFORCE \
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID \
    --project=CLUSTER_PROJECT_ID

Windows (PowerShell)

gcloud beta container clusters create CLUSTER_NAME `
    --location=LOCATION `
    --binauthz-evaluation-mode=POLICY_BINDINGS_AND_PROJECT_SINGLETON_POLICY_ENFORCE `
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID `
    --project=CLUSTER_PROJECT_ID

Windows (cmd.exe)

gcloud beta container clusters create CLUSTER_NAME ^
    --location=LOCATION ^
    --binauthz-evaluation-mode=POLICY_BINDINGS_AND_PROJECT_SINGLETON_POLICY_ENFORCE ^
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID ^
    --project=CLUSTER_PROJECT_ID

Update a cluster to use CV monitoring

In this section, you update a cluster to use CV monitoring with check-based platform policies only. If the cluster already has project-singleton policy enforcement enabled, running this command disables it. Instead, consider updating the cluster with enforcement and CV monitoring enabled.

Before using any of the command data below, make the following replacements:

  • CLUSTER_NAME: the cluster name
  • LOCATION: the location—for example: us-central1 or asia-south1
  • POLICY_PROJECT_ID: the ID of the project where the policy is stored
  • POLICY_ID: the policy ID
  • CLUSTER_PROJECT_ID: the cluster project ID

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud beta container clusters update CLUSTER_NAME \
    --location=LOCATION \
    --binauthz-evaluation-mode=POLICY_BINDINGS \
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID \
    --project=CLUSTER_PROJECT_ID

Windows (PowerShell)

gcloud beta container clusters update CLUSTER_NAME `
    --location=LOCATION `
    --binauthz-evaluation-mode=POLICY_BINDINGS `
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID `
    --project=CLUSTER_PROJECT_ID

Windows (cmd.exe)

gcloud beta container clusters update CLUSTER_NAME ^
    --location=LOCATION ^
    --binauthz-evaluation-mode=POLICY_BINDINGS ^
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID ^
    --project=CLUSTER_PROJECT_ID

Update a cluster to use enforcement and CV monitoring

In this section, you update a cluster to use both project-singleton policy enforcement and CV monitoring with check-based platform policies.

Before using any of the command data below, make the following replacements:

  • CLUSTER_NAME: a cluster name
  • LOCATION: the location—for example: us-central1 or asia-south1
  • POLICY_PROJECT_ID: the ID of the project where the policy is stored
  • POLICY_ID: the policy ID
  • CLUSTER_PROJECT_ID: the cluster project ID

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud beta container clusters update CLUSTER_NAME \
    --location=LOCATION \
    --binauthz-evaluation-mode=POLICY_BINDINGS_AND_PROJECT_SINGLETON_POLICY_ENFORCE \
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID \
    --project=CLUSTER_PROJECT_ID

Windows (PowerShell)

gcloud beta container clusters update CLUSTER_NAME `
    --location=LOCATION `
    --binauthz-evaluation-mode=POLICY_BINDINGS_AND_PROJECT_SINGLETON_POLICY_ENFORCE `
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID `
    --project=CLUSTER_PROJECT_ID

Windows (cmd.exe)

gcloud beta container clusters update CLUSTER_NAME ^
    --location=LOCATION ^
    --binauthz-evaluation-mode=POLICY_BINDINGS_AND_PROJECT_SINGLETON_POLICY_ENFORCE ^
    --binauthz-policy-bindings=name=projects/POLICY_PROJECT_ID/platforms/gke/policies/POLICY_ID ^
    --project=CLUSTER_PROJECT_ID

View logs for CV entries

You can search Cloud Logging entries to find CV configuration errors and CV platform policy validation violations.

CV logs errors and violations to Cloud Logging within 24 hours. You can usually see entries within a few hours.

View CV configuration error logs

To view CV configuration error logs, run the following command:

gcloud logging read \
     --order="desc" \
     --freshness=7d \
     --project=CLUSTER_PROJECT_ID \
    'logName:"binaryauthorization.googleapis.com%2Fcontinuous_validation" "configErrorEvent"'

The following output shows a configuration error in which a CV platform policy isn't found:

{
  "insertId": "141d4f10-72ea-4a43-b3ec-a03da623de42",
  "jsonPayload": {
    "@type": "type.googleapis.com/google.cloud.binaryauthorization.v1beta1.ContinuousValidationEvent",
    "configErrorEvent": {
      "description": "Cannot monitor cluster 'us-central1-c.my-cluster': Resource projects/123456789/platforms/gke/policies/my-policy does not exist."
    }
  },
  "resource": {
    "type": "k8s_cluster",
    "labels": {
      "cluster_name": "my-cluster",
      "location": "us-central1-c",
      "project_id": "my-project"
    }
  },
  "timestamp": "2024-05-28T15:31:03.999566Z",
  "severity": "WARNING",
  "logName": "projects/my-project/logs/binaryauthorization.googleapis.com%2Fcontinuous_validation",
  "receiveTimestamp": "2024-05-28T16:30:56.304108670Z"
}

View CV platform policy validation violations

If no images violate the platform policies that you have enabled, no entries appear in the logs.

To view CV log entries for the last seven days, run the following command:

gcloud logging read \
     --order="desc" \
     --freshness=7d \
     --project=CLUSTER_PROJECT_ID \
    'logName:"binaryauthorization.googleapis.com%2Fcontinuous_validation" "policyName"'

Replace CLUSTER_PROJECT_ID with the cluster project ID.

Check types

CV logs check violation information to checkResults. In the entry, the value checkType indicates the check. The values for each check are as follows:

  • ImageFreshnessCheck
  • SigstoreSignatureCheck
  • SimpleSigningAttestationCheck
  • SlsaCheck
  • TrustedDirectoryCheck
  • VulnerabilityCheck

Example log

The following example CV Logging entry describes a non-conformant image that violates a trusted directory check:

{
  "insertId": "637c2de7-0000-2b64-b671-24058876bb74",
  "jsonPayload": {
    "podEvent": {
      "endTime": "2022-11-22T01:14:30.430151Z",
      "policyName": "projects/123456789/platforms/gke/policies/my-policy",
      "images": [
        {
          "result": "DENY",
          "checkResults": [
            {
              "explanation": "TrustedDirectoryCheck at index 0 with display name \"My trusted directory check\" has verdict NOT_CONFORMANT. Image is not in a trusted directory",
              "checkSetName": "My check set",
              "checkSetIndex": "0",
              "checkName": "My trusted directory check",
              "verdict": "NON_CONFORMANT",
              "checkType": "TrustedDirectoryCheck",
              "checkIndex": "0"
            }
          ],
          "image": "gcr.io/my-project/hello-app:latest"
        }
      ],
      "verdict": "VIOLATES_POLICY",
      "podNamespace": "default",
      "deployTime": "2022-11-22T01:06:53Z",
      "pod": "hello-app"
    },
    "@type": "type.googleapis.com/google.cloud.binaryauthorization.v1beta1.ContinuousValidationEvent"
  },
  "resource": {
    "type": "k8s_cluster",
    "labels": {
      "project_id": "my-project",
      "location": "us-central1-a",
      "cluster_name": "my-test-cluster"
    }
  },
  "timestamp": "2022-11-22T01:44:28.729881832Z",
  "severity": "WARNING",
  "logName": "projects/my-project/logs/binaryauthorization.googleapis.com%2Fcontinuous_validation",
  "receiveTimestamp": "2022-11-22T03:35:47.171905337Z"
}

Clean up

This section describes how to clean up the CV monitoring you configured earlier in this guide.

You can disable CV monitoring or both Binary Authorization and CV in your cluster.

Disable Binary Authorization in a cluster

To disable both CV and Binary Authorization enforcement in your cluster, run the following command:

gcloud beta container clusters update CLUSTER_NAME \
    --binauthz-evaluation-mode=DISABLED \
    --location=LOCATION \
    --project=CLUSTER_PROJECT_ID

Replace the following:

  • CLUSTER_NAME: the name of the cluster
  • LOCATION: the cluster location
  • CLUSTER_PROJECT_ID: the cluster project ID

Disable check-based policy monitoring in a cluster

To disable CV with check-based policies in the cluster, and re-enable enforcement using the Binary Authorization enforcement policy, run the following command:

gcloud beta container clusters update CLUSTER_NAME  \
    --binauthz-evaluation-mode=PROJECT_SINGLETON_POLICY_ENFORCE \
    --location=LOCATION \
    --project="CLUSTER_PROJECT_ID"

Replace the following:

  • CLUSTER_NAME: the name of the cluster
  • LOCATION: the cluster location
  • CLUSTER_PROJECT_ID: the cluster project ID

Note that --binauthz-evaluation-mode=PROJECT_SINGLETON_POLICY_ENFORCE is equivalent to the older flag --enable-binauthz.

Delete the policy

To delete the policy, run the following command. It is not necessary to delete the check-based platform policy to disable check-based policy auditing.

gcloud beta container binauthz policy delete POLICY_ID \
    --platform=gke \
    --project="POLICY_PROJECT_ID"

Replace the following:

  • POLICY_ID: the ID of the policy
  • POLICY_PROJECT_ID: the policy project ID

What's next