Hi all, my flink job consume kafka topic A, and write to kafka topic B.
When i restart my flink job via savepoint, topic B have some duplicate
message. Any one can help me how to solve this problem? Thanks!
My Versions:
Flink 1.12.4
Kafka 2.0.1
Java 1.8
Core code:
env.enableCheckpointing(30);
I restart flink job via savepoint. command as following:
cancel command:
/home/datadev/flink-1.12.2/flink-1.12.2/bin/flink cancel \
-yid application_1625497885855_698371 \
-s
hdfs://ztcluster/flink_realtime_warehouse/checkpoint/UserClickLogAll/savepoint
\
59cf6ccc83aa163bd1e0cd3304dfe06a
print s
to continue 😊
… let’s stay on the mailing list to give others a chance to reply
Matthias
From: Jim Chen
Sent: Montag, 2. August 2021 10:56
To: Schwalbe Matthias
Subject: Re: Any one can help me? Flink consume Kafka duplicate message via
savepoint restart
Hi Schwalbe,
The dulplicate message
