We recommend using the following tools to assist you in various stages of your migration, depending on your source database and other factors. Some tools only support certain source databases. For some steps of the migration process, no tool is available, so you must complete those steps manually.
Spanner migration tool (SMT) is an open-source tool that performs assessments, schema conversion, and data migrations. For information on setting up the Spanner migration tool, see Set up Spanner migration tool.
Datastream is a Google Cloud service that lets you read change data capture (CDC) events and bulk data from a source database and write to a specified destination.
Bulk data migration is a Dataflow template that lets you migrate large MySQL data sets directly to Spanner.
Live data migration uses Datastream and Dataflow to migrate:
- Existing data in your source database.
- Stream of changes that are made to your source database during the migration.
Data Validation Tool (DVT) is a standardized data validation method built by Google and supported by the open source community. You can integrate DVT into existing Google Cloud products.
Database Migration Assessment (DMA) offers a basic assessment to migrate MySQL and PostgreSQL to Spanner.
Migration tools for MySQL source databases
If your source database is MySQL, then you can perform some of the initial migration stages using MySQL dump files. You must directly connect to your running source MySQL database to complete a production migration.
The following table recommends migration tools based on the migration stage and whether you're using a dump file or directly connecting your source database:
| Migration stage | Dump file | Direct connection to source database |
|---|---|---|
| Assessment |
Use SMT
with mysqldump.
|
Use SMT
with mysqldump.
|
| Schema conversion |
Use SMT
with mysqldump.
|
Use SMT to configure and convert schema. |
| Sample data load |
|
Perform a bulk migration. |
| Data migration | Not applicable | Perform a bulk migration, then perform a minimal downtime migration. |
| Data validation | Not applicable | Use DVT. |
| Cutover and fallback configuration | Not applicable | Use SMT for reverse replication. |
Migration tools for PostgreSQL source databases
If your source database uses PostgreSQL, then you can perform some of the migration stages using a PostgreSQL dump file. You must directly connect to your running source PostgreSQL database to complete the migration.
The following table recommends migration tools based on the migration stage and whether you're working with a dump file or directly connecting from your source database:
| Migration stage | Dump file | Direct connection to source database |
|---|---|---|
| Assessment | Use SMT with pg_dump. |
Use DMA. |
| Schema conversion | Use SMT with pg_dump. |
Use SMT to configure and convert schema. |
| Sample data load |
|
Perform a minimal downtime migration. |
| Data migration | Not applicable | Perform a minimal downtime migration. |
| Data validation | Not applicable | Use DVT. |
| Cutover and fallback configuration | Not applicable | Not applicable |
Migration tools for a Cassandra-source databases
Spanner offers a Cassandra-compatible interface that supports near-zero application code changes when migrating from Cassandra to Spanner. For more information about compatibility details, see the Cassandra overview.
The following table recommends migration tools based on the migration stage:
| Migration stage | Recommended tool or process |
|---|---|
| Assessment | Not applicable |
| Schema conversion | Schema conversion tool |
| Data migration |
|
| Data validation |
For large scale databases (>10 million rows): build your own tooling. For more information, see Tips to validate Cassandra using row matching. For small scale databases (<10 million rows), see the sample validation scripts in GitHub. |
| Failover configuration | Use the Spanner migration tool CLI (SMT) in GitHub for reverse replication. |