[
https://issues.apache.org/jira/browse/FLINK-37860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanquan Lv closed FLINK-37860.
------------------------------
Fix Version/s: cdc-3.5.0
Assignee: MOBIN
Resolution: Fixed
Fixed in master via 2fcf6f216aa74da4e0309d261e95f3462c2d1cc5
> [paimon-connector]Fix Committer.createContext not passing OperatorStateStore
> correctly
> --------------------------------------------------------------------------------------
>
> Key: FLINK-37860
> URL: https://issues.apache.org/jira/browse/FLINK-37860
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.4.0
> Reporter: MOBIN
> Assignee: MOBIN
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.5.0
>
>
> pipeline.yml
> {code:java}
> source:
> ....
> sink:
> type: paimon
> name: Paimon Sink
> catalog.properties.metastore: hive
> catalog.properties.uri: thrift://test:9083
> catalog.properties.warehouse: hdfs://test/warehouse/
> table.properties.metastore.partitioned-table: true
> ...{code}
> The metastore.partitioned-table parameter of paimon will store partition
> information in the state and trigger updates to HMS periodically. When the
> user turns on this parameter, NEP exception will occur because the
> OperatorStateStore is not correctly passed in Committer.createContext.
> {code:java}
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at
> org.apache.paimon.flink.sink.StoreCommitter.<init>(StoreCommitter.java:63)
> at
> org.apache.paimon.flink.sink.StoreMultiCommitter.getStoreCommitter(StoreMultiCommitter.java:204)
> at java.util.HashMap$KeySet.forEach(HashMap.java:928)
> at
> org.apache.paimon.flink.sink.StoreMultiCommitter.commit(StoreMultiCommitter.java:128)
> at
> org.apache.flink.cdc.connectors.paimon.sink.v2.PreCommitOperator.snapshotState(PreCommitOperator.java:116)
> at
> org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:234)
> ... 33 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.paimon.flink.sink.partition.ReportPartStatsListener.<init>(ReportPartStatsListener.java:75)
> at
> org.apache.paimon.flink.sink.partition.ReportPartStatsListener.create(ReportPartStatsListener.java:180)
> at
> org.apache.paimon.flink.sink.partition.PartitionListeners.create(PartitionListeners.java:62)
> at
> org.apache.paimon.flink.sink.StoreCommitter.<init>(StoreCommitter.java:61)
> ... 38 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
