This section provides troubleshooting guidance for common issues encountered when using a Cloud Storage Sink connector.
Deserialization is misconfigured
If the connector's converters don't match the data in Kafka, the connector fails with an error message similar to the following:
org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka
Connect data failed due to serialization error:
To resolve this issue, make sure the key and value converters for the connector match the data that the connector is reading from Kafka.
For example, if you set the format output type to csv and the value converter
to JsonConverter, you might see the following error:
org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka
Connect data failed due to serialization error:
at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:333)
at org.apache.kafka.connect.storage.Converter.toConnectData(Converter.java:91)
This error occurs because CSV output requires ByteArrayConverter for the value
converter, not JsonConverter. For more information, see
How a Cloud Storage Sink connector works.
The stack trace in the logs might help to locate the root cause. For example, the following log entry indicates a problem converting a UTF-32 character:
Caused by: org.apache.kafka.common.errors.SerializationException:
java.io.CharConversionException: Invalid UTF-32 character 0x31d086a
(above 0x0010ffff) at char #1, byte #7
Missing Cloud Storage bucket permissions
If the Managed Kafka service account doesn't have permission to write to the Cloud Storage bucket, the connector fails with an error message similar to the following:
org.apache.kafka.connect.errors.ConnectException:
com.google.cloud.storage.StorageException: 403 Forbidden
To resolve this issue, grant the Storage Object Creator role
(roles/storage.objectCreator) to the Managed Kafka service account. For more
information, see
Grant permissions to write to the Cloud Storage bucket.