Chengbing Liu created FLINK-39197:
-------------------------------------
Summary: Fix NPE when finding chunk end
Key: FLINK-39197
URL: https://issues.apache.org/jira/browse/FLINK-39197
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.5.0
Reporter: Chengbing Liu
We encountered the following problem when using MySQL pipeline connector:
{noformat}
Caused by: org.apache.flink.util.FlinkRuntimeException: Chunk splitting has
encountered exception
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:698)
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:398)
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:151)
at
org.apache.flink.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:223)
at
org.apache.flink.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
at
org.apache.flink.runtime.source.coordinator.SourceCoordinator.handleRequestSplitEvent(SourceCoordinator.java:568)
at
org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$3(SourceCoordinator.java:295)
at
org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$10(SourceCoordinator.java:469)
... 8 more
Caused by: java.lang.IllegalStateException: Error when splitting chunks for
db1.table1
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:359)
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:676)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Caused by: java.lang.NullPointerException
at
org.apache.flink.cdc.connectors.mysql.source.utils.ObjectUtils.compare(ObjectUtils.java:88)
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.nextChunkEnd(MySqlChunkSplitter.java:360)
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitOneUnevenlySizedChunk(MySqlChunkSplitter.java:176)
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:138)
at
org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:357)
... 6 more
{noformat}
This should be because max values are removed resulting in {{chunkEnd}} ==
null, as mentioned in
{noformat}
// chunk end might be null when max values are removed
Object chunkEnd =
queryNextChunkMax(jdbc, tableId, splitColumn, chunkSize,
previousChunkEnd);
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
