Scheduling backups in Cloud Storage

This page describes how to schedule backups for Cassandra in Cloud Storage. In this method, backups are stored in the specified Cloud Storage bucket.

To schedule Cassandra backups, perform the following steps:

Run the following create-service-account command to create a Google Cloud service account (SA) with the standard roles/storage.objectAdmin role. This SA role allows you to write backup data to Cloud Storage. Execute the command in the $APIGEE_HELM_CHARTS_HOME/apigee-operator/etc/ directory.

./tools/create-service-account --env non-prod --dir ./

This command creates a single service account named apigee-non-prod for use in non-production environments and places the downloaded key file in the ./ directory.

Note: If you prefer to create all the individual service accounts for a production environment, use the following command:

./tools/create-service-account --env prod --dir ./

Then, move each service account into its corresponding chart directory. The following table lists each service account and the corresponding Apigee hybrid Helm charts:

Service account	Apigee Helm chart
`apigee-cassandra`	`apigee-datastore`
`apigee-logger`	`apigee-telemetry`
`apigee-mart`	`apigee-org`
`apigee-metrics`	`apigee-telemetry`
`apigee-runtime`	`apigee-env`
`apigee-synchronizer`	`apigee-env`
`apigee-udca`	`apigee-org`
`apigee-watcher`	`apigee-org`

For more information about Google Cloud service accounts, see Creating and managing service accounts.

The create-service-account command saves a JSON file containing the service account private key. The file is saved in the same directory where the command executes. You will need the path to this file in the following steps.
Create a Cloud Storage bucket. Specify a reasonable data retention policy for the bucket. Apigee recommends a data retention policy of 15 days.
Open your overrides.yaml file.

Add the following cassandra.backup properties to enable backup. Do not remove any of the properties that are already configured.

Parameters

cassandra:
    ...

    backup:
      enabled: true
      serviceAccountPath: SA_JSON_FILE_PATH
      dbStorageBucket: CLOUD_STORAGE_BUCKET_NAME
      schedule: BACKUP_SCHEDULE_CODE
      cloudProvider: "GCP"  # For remote server backup set this to HYBRID (all caps)

    ...

Example

  ...

  cassandra:
    storage:
      type: gcepd
      capacity: 50Gi
      gcepd:
        replicationType: regional-pd
    auth:
      default:
        password: "abc123"
      admin:
        password: "abc234"
      ddl:
        password: "abc345"
      dml:
        password: "abc456"
    nodeSelector:
      key: cloud.google.com/gke-nodepool
      value: apigee-data
    backup:
      enabled: true
      serviceAccountPath: "/Users/myhome/.ssh/my-cassandra-backup-sa.json"
      dbStorageBucket: "myname-cassandra-backup"
      schedule: "45 23 * * 6"
      cloudProvider: "GCP"
      


    ...

Property	Description
`backup:enabled`	Backup is disabled by default. You must set this property to `true`.
`backup:serviceAccountPath`	`SA_JSON_FILE_PATH` The path on your filesystem to the service account JSON file that was downloaded when you ran the `create-service-account` command. You can also provide a relative file path. The path will be relative to the `hybrid-base-directory/hybrid-files` directory.
`backup:dbStorageBucket`	`CLOUD_STORAGE_BUCKET_NAME` The name of an existing Google Cloud Storage bucket that will be used to store backup archives. See Creating buckets if you need to create one. Note: Starting in Hybrid version 1.13, the bucket name doesn't need to be prefixed with `gs://`. Support for this prefix has been deprecated and will be removed in future releases.
`backup:cloudProvider`	`GCP/HYBRID` For a Cloud Storage backup, set the property to `GCP`. For example, `cloudProvider: "GCP"`. For a remote server backup, set the property to `HYBRID`. For example, `cloudProvider: "HYBRID"`.
`backup:schedule`	`BACKUP_SCHEDULE_CODE` The time when the backup starts, specified in standard crontab syntax. Default: `0 2 * * ` Note:* Avoid scheduling a backup that starts a short time after you apply the backup configuration to your cluster. When you apply the backup configuration, Kubernetes recreates the Cassandra nodes. If the backup starts before the nodes restart, the first backup will fail and the subsequent backups will pass.

Apply the configuration changes to the new cluster. For example:
```
helm upgrade datastore apigee-datastore/ \
  --namespace APIGEE_NAMESPACE \
  --atomic \
  -f OVERRIDES_FILE.yaml
```
Note: If you see an error saying Error: UPGRADE FAILED: "datastore" has no deployed releases, replace upgrade with install and try the command again.

Where OVERRIDES_FILE is the path to the overrides file you just edited.

Verify the backup job. For example:

kubectl get cronjob -n APIGEE_NAMESPACE

  NAME                      SCHEDULE     SUSPEND   ACTIVE   LAST SCHEDULE   AGE
  apigee-cassandra-backup   33 * * * *   False     0        <none>          94s

Launch a manual backup

Cassandra backups generate automatically according to the cron schedule set in the overrides.yaml file.

To initiate a manual backup, use this command:

kubectl create job -n APIGEE_NAMESPACE --from=cronjob/apigee-cassandra-backup BACKUP_POD_NAME