Apigee hybrid Helm migration tool

This migration tool assists in migrating an apigeectl-based hybrid cluster to a Helm-based hybrid cluster. This tool performs no actual replacement of any cluster components. It is idempotent and can be run many times on the same cluster, preparing a subset of components and orgs each time.

You can migrate all apigee components at one time, and Helm upgrade operations can be done on a per-component basis after the tool is run.

See Installing and managing Apigee hybrid with Helm charts for information about managing hybrid clusters you have migrated to Helm management with this tool.

Prerequisites

Helm version 3.10+.
A working kubeconfig file pointing to a cluster with a working Apigee hybrid 1.10 installation.
Permissions to modify the metadata and annotations on the Kubernetes resources of the hybrid components that you wish to migrate.

Scope

This tool supports the following options at run-time:

Customization of namespace for apigee resources. Default namespace: apigee
Migration of selected hybrid components only. Default: all components are migrated
Migration of a single org only
Migration of a single env only
Migration of a single env-group (apigee-virtualhost) only
Customization of Helm release names for orgs, envs, and env-groups.

Limitations

This tool does not support customizing Helm release names for these hybrid components: apigee-operator, apigee-datastore, apigee-redis, apigee-telemetry, and apigee-ingress-manager.
Interactive customizations made to the Helm release names for orgs, envs, and env-groups do not automatically persist between runs. You can edit the temp file and supply it as an option in later runs.
Env and env-group filtering is done by name only. In some cases this could result in multiple envs and env-groups being migrated on multi-org clusters.

For example, on a multi-org cluster with orgs org1 and org2, if the env prod is present in both orgs, and only --env=prod is specified, both envs will be migrated. If you wish to only migrate a single env, you must also specify an org filter --org=org1 or --org=org2.

Usage

Syntax

apigee-helm-migration [--apigee-namespace=] [--components=] [--dry-run] [--env=org1] [--env-group=org2] [--org=baz] [--kubeconfig=] [-y] [-v] [-f /path/to/releaseNames.yaml]

Generated Helm release names

Every Helm Chart that is deployed on a cluster needs to have a release name, which must be unique within a namespace. Helm release names don't have any naming convention or restriction relative to the chart name. The migration tool generates unique Helm release names for every component.

Chart	Single-org cluster	Multi-org cluster
`apigee-operator`	`operator`	`operator`
`apigee-datastore`	`datastore`	`datastore`
`apigee-telemetry`	`telemetry`	`telemetry`
`apigee-redis`	`redis`	`redis`
`apigee-ingress-manager`	`ingress-manager`	`ingress-manager`
`apigee-org`	`ORG_NAME`	`ORG_NAME`
`apigee-env`	`ENV_NAME[-env[-n]]⁽¹⁾`	`ORG_NAME-ENV_NAME[-env[-n]]`⁽¹⁾
`apigee-virtualhost (envgroup)`	`VH_NAME[-env-group[-n]]`⁽¹⁾	`ORG_NAME-VH_NAME[-env-group[-n]]`⁽¹⁾
⁽¹⁾ Names are suffixed with `-env` or `-env-group` if the generated name conflicts with another generated name. They are further suffixed with `-1` or `-2 …` if they still conflict.

Customizing Helm release names

The migration tool allows interactive customization of Helm release names. If you want to customize the Helm release names non-interactively:

Run the tool once and exit at the first prompt to create a temporary file containing the auto-generated release names. You should see a line like:
```
INFO: 21:32:56 using temp file for release names:  /tmp/apigee-helm-migration-1229129207-releaseNames
```

Move or copy and then edit this file. You can pass this edited file in with the -f option when you run the migration tool. The auto-generated release names look like:

orgs:
  example-apigee-org:
    helmReleaseName: example-apigee-org
    envs:
      prod:
        helmReleaseName: prod
    envGroups:
      prod-envgroup:
        helmReleaseName: prod-envgroup

To customize the Helm release names for an org, env, or envgroup, edit the helmReleaseName field of that object. For example, to rename the org release to custom-org, the env release to custom-env, and the envgroup release to custom-group, the resulting file looks like:

orgs:
  example-apigee-org:
    helmReleaseName: custom-org
    envs:
      prod:
        helmReleaseName: custom-env
    envGroups:
      prod-envgroup:
        helmReleaseName: custom-group

Using custom namespaces

Apigee hybrid runs in two Kubernetes namespaces:

apigee-system: The apigee-operator component always runs in the apigee-system namespace. The Helm migration tool will update the apigee-operator component in the apigee-system namespace regardless of what you specify with the --apigee-namespace flag.
apigee: All hybrid components except apigee-operator run in this namespace. apigee is the default name. You can use any custom namespace for these components.
If you use a custom namespace, you must specify the it with the --apigee-namespace my_custom_namespace flag when you run the Helm migration tool.

You must also add the namespace: my_custom_namespace top-level property to your overrides file.

Directions

Download the migration tool.
Linux
1. Store the latest version number in a variable using the following command:
  export VERSION=$(curl -s "https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/current-version.txt?ignoreCache=1")
2. Check that the variable was populated with a version number using the following command. If you want to use a different version, you can save that in an environment variable instead.
  echo $VERSION
  For example:
  echo $VERSION 1.0.5
3. Download the release package for your operating system using the following command:
  curl -LO https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/${VERSION}/apigee-helm-migration_linux_64.tar.gz
4. Extract the compressed files using the following command:
  tar -xzf apigee-helm-migration_linux_64.tar.gz
Mac OS
1. Store the latest version number in a variable using the following command:
  export VERSION=$(curl -s "https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/current-version.txt?ignoreCache=1")
2. Check that the variable was populated with a version number using the following command. If you want to use a different version, you can save that in an environment variable instead.
  echo $VERSION
  For example:
  echo $VERSION 1.0.5
3. Download the release package for your operating system using the following command:
  curl -LO https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/${VERSION}/apigee-helm-migration_mac_64.tar.gz
4. Extract the compressed files using the following command:
  tar -xzf apigee-helm-migration_mac_64.tar.gz
Windows
1. Store the latest version number in a variable using the following command:
  for /f "tokens=*" %a in ('curl -s https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/current-version.txt') do set VERSION=%a
2. Check that the variable was populated with a version number using the following command. If you want to use a different version, you can save that in an environment variable instead.
  echo %VERSION%
  For example:
  echo %VERSION% 1.0.5
3. Download the release package for your operating system using the following command:
  curl -LO https://storage.googleapis.com/apigee-release/hybrid/apigee-migration-tool/%VERSION%/apigee-helm-migration_windows_64.tar.gz
4. Extract the compressed files using the following command:
  tar xzvf apigee-helm-migration_windows_64.tar.gz

Run the migration tool. If the default options are acceptable, it is sufficient to run the tool without any arguments, and approve the prompt if the generated helm release names are satisfactory. Some example scenarios are given below:

A simple installation, using the default kubeconfig (~/.kube/config), default apigee namespace, and default Helm release names.

The following command should be sufficient for most, if not all, installations. Helm upgrade operations can be done on a per-component basis after the tool is run.
```
./apigee-helm-migration
```

Migrating all components using a custom namespace:

./apigee-helm-migration --apigee-namespace my_custom_namespace

Migrating only the operator and datastore components:

./apigee-helm-migration --components operator,datastore

  INFO: 00:22:48 using kubeconfig file  /usr/local/google/home/example/.kube/config
  INFO: 00:22:48 namespace for apigee resources:
  INFO: 00:22:48 	 apigee
  INFO: 00:22:48 processing all organizations in cluster
  INFO: 00:22:48 Components to migrate:
  INFO: 00:22:48 	 operator,datastore
  INFO: 00:22:48 dry-run:
  INFO: 00:22:48 	 false
  Continue with patching apigee resources for Helm migration? [y/n]: y
  INFO: 00:22:52 Processing component:  operator
  INFO: 00:22:54 Processing component:  datastore
  INFO: 00:22:55 Migration successful!

Pointing to a specific kubeconfig file and specifying a different name for the apigee namespace.
```
./apigee-helm-migration --kubeconfig /abs/path/to/kubeconf --namespace org1_namespace
```
Migrating all components, but only a single org:
```
./apigee-helm-migration --org=some-test-org
```

The following shows an example output from a successful migration:

INFO: 21:32:55 using kubeconfig file  /usr/local/google/home/example/.kube/config
INFO: 21:32:55 namespace for apigee resources:
INFO: 21:32:55 	 apigee
INFO: 21:32:55 processing all organizations in cluster
INFO: 21:32:55 processing all components
INFO: 21:32:55 dry-run:
INFO: 21:32:55 	 false
INFO: 21:32:55 cluster Apigee information:
INFO: 21:32:55 Apigee Organizations found:
INFO: 21:32:56 	 example-hybrid-dev
INFO: 21:32:56 Apigee Environments found (org: env):
INFO: 21:32:56 	 example-hybrid-dev : prod
INFO: 21:32:56 Apigee EnvGroups(apigeerouteconfigs) found (org: envGroup):
INFO: 21:32:56 	 example-hybrid-dev : prod-envgroup
INFO: 21:32:56 using temp file for release names:  /tmp/apigee-helm-migration-1229129207-releaseNames
INFO: 21:32:56 Helm release names for Apigee orgs/envs/envgroups:
orgs:
example-hybrid-dev:
helmReleaseName: example-hybrid-dev
envs:
  prod:
    helmReleaseName: prod
envGroups:
  prod-envgroup:
    helmReleaseName: prod-envgroup
Make changes to the release names for Apigee orgs/env/envgroups? [y/n]: n
Continue with patching apigee resources for Helm migration? [y/n]: y
INFO: 21:32:59 Processing component:  operator
INFO: 21:33:01 Processing component:  datastore
INFO: 21:33:01 Processing component:  redis
INFO: 21:33:02 Processing component:  ingress-manager
INFO: 21:33:02 Processing component:  telemetry
INFO: 21:33:03 Processing component:  orgs
INFO: 21:33:05 Processing component:  envs
INFO: 21:33:06 Processing component:  env-groups
INFO: 21:33:07 Migration successful!

Potential errors:

Error parsing release names file: Check the passed in release names file.
Resources not found: Check that Apigee hybrid is fully installed, and you have permissions to access the apigee resources.

Next step

Continue your installation of the Apigee hybrid Helm charts with the instructions in Installing and managing Apigee hybrid with Helm charts.

Important: When following the instructions for your migrated cluster, you must omit the ‑‑atomic flag from any Helm commands to avoid accidentally deleting resources if the helm upgrade command fails.

Output of -help

./apigee-helm-migration --help

Usage of ./apigee-helm-migration:
  -apigee-namespace string
      namespace used for apigee resources (default "apigee")
  -components string
      CSV of components to migrate. If empty then all components are migrated. Valid values are: operator,datastore,redis,ingress-manager,telemetry,orgs,envs,env-groups
  -dry-run
      perform a dry-run
  -env string
      restrict migration to a singular supplied env. If empty then all envs detected in the cluster are migrated
  -env-group string
      restrict migration to a singular supplied envGroup. If empty then all envGroups detected in the cluster are migrated
  -kubeconfig string
      (optional) absolute path to the kubeconfig file (default "/usr/local/google/home/example/.kube/config")
  -org string
      restrict migration to a singular supplied org. If empty then all orgs detected in the cluster are migrated
  -v	Increased logging verbosity
  -y	don't prompt for confirmation or for configuration of Helm releases