Hi Yordan, I don't understand where the problem is, why do you think savepoints are unusable? If you recover with `ignoreFailuresAfterTransactionTimeout` enabled, the current Flink behaviour shouldn't cause any problems (except for maybe some logged errors).
Best, Piotrek wt., 15 lis 2022 o 15:36 Yordan Pavlov <y.d.pav...@gmail.com> napisał(a): > Hi, > we are using Kafka savepoints as a recovery tool and want to store > multiple ones for the past months. However as we use Kafka > transactions for our KafkaSink this puts expiration time on our > savepoints. We can use a savepoint only as old as our Kafka > transaction timeout. The problem is explained in this issue: > https://issues.apache.org/jira/browse/FLINK-16419 > the relative comment being this one: > "FlinkKafkaProducer or KafkaSink do not know during recovery if they > have to recover and commit or if it has already happened. Due to that, > they are always attempting to recover and commit transactions during > startup." > I'm surprised that more people are not hitting this problem as this > makes Savepoints pretty much unusable as a recovery mechanism. >
