Data integrity assurance

This page describes how to use checksums to maintain and verify the integrity of your secret's data when adding and accessing secret versions.

Think of a checksum as a unique fingerprint for your data. It's a short code generated from the secret data using the CRC32C algorithm. If even a single bit in your secret data changes, the checksum also changes. This lets Secret Manager detect any accidental modifications or corruption.

Secret Manager uses checksums in the following ways:

  1. When you add a secret version:

    • Secret Manager calculates the CRC32C checksum of your secret data.

    • This checksum is stored along with the secret data.

  2. When you access a secret version:

    • Secret Manager returns the secret data along with its checksum.

    • You can use this checksum to verify that the data you received is exactly the same as the data stored in Secret Manager.

To ensure that the checksum is compatible with the SecretPayload structure, the checksum of the secret data must be calculated using the CRC32C algorithm and encoded as a decimal integer. The SecretVersion response includes a field indicating whether the server has successfully received and validated this checksum.

The following example shows how checksums work in Secret Manager:

API

These examples use curl to demonstrate using the API. You can generate access tokens with gcloud auth print-access-token. On Compute Engine or GKE, you must authenticate with the cloud-platform scope.

With secret data stored in a data file, calculate the checksum, using gcloud storage hash. Checksum must be converted to decimal format; it is encoded as int64 in SecretPayload proto. 

$ gcloud storage hash "/path/to/file.txt" --hex

With secret data passed on the command line, calculate the checksum as follows: 

$ gcloud storage hash --hex cat <(echo ${SECRET_DATA})

Base64-encode the secret data and save it as a shell variable.

$ SECRET_DATA=$(echo "seCr3t" | base64)

Invoke the API using curl.

$ curl "https://secretmanager.googleapis.com/v1/projects/project-id/secrets/secret-id:addVersion" \
    --request "POST" \
    --header "authorization: Bearer $(gcloud auth print-access-token)" \
    --header "content-type: application/json" \
    --data "{\"payload\": {\"data\": \"${SECRET_DATA}\", \"data_crc32c\": $CHECKSUM}}"

When the secret version is accessed, the returned SecretPayload contains the data along with its checksum. Following is a sample response:

{
  "name": "projects/PROJECT_ID/secrets/SECRET_ID/versions/VERSION_ID",
  "payload": {
    "data": "YQo=",
    "dataCrc32c": "163439259"
  }
}

In the console, when you add a secret version, the checksum is automatically calculated when you enter a value for the secret.

Secret versions encrypted with customer-managed encryption keys (CMEK) and created before July 16, 2021 do not have checksums stored.

What's next