Configure a self-managed MySQL database for CDC

This page describes how to configure change data capture (CDC) to stream data from a self-managed MySQL database to a supported destination, such as BigQuery or Cloud Storage. These instructions also work for databases such as MariaDB, Percona Server for MySQL or Alibaba Cloud PolarDB.

Verify the configuration and retention period of the binary log

1. Confirm that the binary log is configured correctly by entering the following MySQL command:

   SHOW GLOBAL VARIABLES LIKE '%binlog_format%';
   

2. Verify that the value for the binlog_format variable is set to ROW. Datastream doesn't support MIXED or STATEMENT binary log formats.

3. Confirm that the row format for the binary log is set to FULL by entering the following MySQL command:

   SHOW GLOBAL VARIABLES LIKE 'binlog_row_image';
   

4. Verify that the replica updates option for the binary log is set to ON by entering the following MySQL command:

   SHOW GLOBAL VARIABLES LIKE 'log_replica_updates';
   

   For MySQL versions earlier than 8.0.26, enter the following command instead:

   SHOW GLOBAL VARIABLES LIKE 'log_slave_updates';
   

5. Verify that the retention period of the binary log is set to 604800 seconds (7 days) by entering the following MySQL command:

   SHOW GLOBAL VARIABLES LIKE 'binlog_expire_logs_seconds';
   

   For MySQL versions earlier than 8.0.3, enter the following command instead:

   SHOW GLOBAL VARIABLES LIKE 'expire_logs_days';
   

Create a Datastream user

1. To create a Datastream user, enter the following MySQL commands:

   CREATE USER 'datastream'@'%' IDENTIFIED BY 'YOUR_PASSWORD';
   GRANT REPLICATION SLAVE, SELECT, REPLICATION CLIENT ON *.* TO 'datastream'@'%';
   FLUSH PRIVILEGES;

What's next

• Learn more about how Datastream works with MySQL sources.