Re: Kafka Sink and Kafka transaction timeout

2023-10-02 Thread Tzu-Li (Gordon) Tai
Hi Lorenzo, The main failure scenario that recommendation is addressing is when the Flink job fails right after a checkpoint successfully completes, but before the KafkaSink subtasks receive from the JM the checkpoint completed RPC notification to commit the transactions. It is possible that

Kafka Sink and Kafka transaction timeout

2023-10-02 Thread Lorenzo Nicora
Hi team In Kafka Sink docs [1], with EXACTLY_ONCE it is recommended to set: transaction_timeout > maximum_checkpoint duration + maximum_restart_duration. I understand transaction_timeout > maximum_checkpoint_duration But why adding maximum_restart_duration? If the application recovers from a