[jira] [Created] (FLINK-34764) FlinkCDC is incompatible with MariaDB 10.0.24

[Flink CDC Issue Import (Jira)]

Flink CDC Issue Import created FLINK-34764:
----------------------------------------------

             Summary: FlinkCDC is incompatible with MariaDB 10.0.24
                 Key: FLINK-34764
                 URL: https://issues.apache.org/jira/browse/FLINK-34764
             Project: Flink
          Issue Type: Bug
          Components: Flink CDC
            Reporter: Flink CDC Issue Import


FlinkCDC is incompatible with MariaDB 10.0.24.

There will be **exceptions** when **debezium** parses the binlog.

mysqlbinlog fails to read binlog event which inserts **TIME, DATETIME and 
TIMESTAMP** with wrong number of microseconds.

The low-level temporal format used by TIME, DATETIME and TIMESTAMP is different 
in MySQL 5.6 and [MariaDB 10.0|https://mariadb.com/kb/en/what-is-mariadb-100/]. 
(In [MariaDB 10.1|https://mariadb.com/kb/en/what-is-mariadb-101/], the MySQL 
implementation is used by default - see 
[mysql56_temporal_format|https://mariadb.com/kb/en/server-system-variables/#mysql56_temporal_format).)

**Environment :**
 - Flink version :  1.13.6
 - Flink CDC version: 2.2.1
 - Database and version: MariaDB 10.0.24

**To Reproduce**
Steps to reproduce the behavior:
1. The test code :
The field type in the database is **Datetime(6)**, and the value is 
**2016-12-21 17:57:16.000000**. 
After analysis, the obtained string is **103946-26-59 06:40:65**.
2. The error :
There will be exceptions when debezium parses the binlog:

`java.io.EOFException: Failed to read next byte from position -1536265298
        at 
com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:213)
        at 
com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readInteger(ByteArrayInputStream.java:52)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeLong(AbstractRowsEventDataDeserializer.java:261)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeCell(AbstractRowsEventDataDeserializer.java:166)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:145)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserializeRows(UpdateRowsEventDataDeserializer.java:72)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:58)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:33)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:329)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:238)
        at 
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
        at 
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:949)
        at 
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:599)
        at 
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:854)
        at java.lang.Thread.run(Thread.java:748)
java.io.EOFException: Failed to read next byte from position -1536265298
        at 
com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:213)
        at 
com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readInteger(ByteArrayInputStream.java:52)
        at 
io.debezium.connector.mysql.RowDeserializers.deserializeVarString(RowDeserializers.java:298)
        at 
io.debezium.connector.mysql.RowDeserializers$UpdateRowsDeserializer.deserializeVarString(RowDeserializers.java:130)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeCell(AbstractRowsEventDataDeserializer.java:194)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:145)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserializeRows(UpdateRowsEventDataDeserializer.java:72)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:58)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.UpdateRowsEventDataDeserializer.deserialize(UpdateRowsEventDataDeserializer.java:33)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:329)
        at 
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:238)
        at 
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
        at 
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:949)
        at 
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:599)
        at 
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:854)
        at java.lang.Thread.run(Thread.java:748]
`

<img width="946" alt="image" 
src="https://user-images.githubusercontent.com/35026750/210321758-45048701-d93c-488f-ac32-2321180e50d9.png";>



**Additional Description**

[MariaDB 10.1.2|https://mariadb.com/kb/en/mariadb-1012-release-notes/] 
introduced the 
[--mysql56-temporal-format|https://mariadb.com/kb/en/server-system-variables/#mysql56_temporal_format]
 option, on by default, which allows MariaDB to store DATETMEs using the same 
low-level format MySQL 5.6 uses. For more information, see [Internal 
Format|https://mariadb.com/kb/en/datetime/#internal-format], below.

[incompatibilities between MariaDB 10.0 and MySQL 
5.6|https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility]
[mysql56_temporal_format|https://mariadb.com/kb/en/server-system-variables/#mysql56_temporal_format]
[DATETIME|https://mariadb.com/kb/en/datetime]
[mysqlbinlog fails to read binlog event which inserts timestamp with wrong 
number of microseconds|https://jira.mariadb.org/browse/MDEV-9567]
[MariaDB 10.0.25 Datetime 类型无法解析毫秒级|https://github.com/alibaba/canal/issues/385]


---------------- Imported from GitHub ----------------
Url: https://github.com/apache/flink-cdc/issues/1851
Created by: [lollipopcs|https://github.com/lollipopcs]
Labels: bug, 
Created at: Tue Jan 03 16:12:57 CST 2023
State: open




--
This message was sent by Atlassian Jira
(v8.20.10#820010)