latency related to the checkpointing mode EXACTLY ONCE

Hi, We use the checkpointing mode EXACTLY ONCE for some of our flink jobs. I wonder how the checkpoint configurations specially its checkpoint interval are related to the end to end latency. We need to setup read_commit true for the kafak consumers. Does this lead a latency from one flink job

RE: latency related to the checkpointing mode EXACTLY ONCE

Many thanks for your quick response. The config read_commit for the kafka consumers is required by the exactly once (EOS)? No exactly once if we read un committed messages? Regards, Min From: Chesnay Schepler Sent: Thursday, February 18, 2021 8:27 PM To: Tan, Min ; user Subject: [External

Reading Flink states from svaepoint uning State Processor API

Hi, I am using Flink 1.10.1 and try to read the flink states from a savepoint using Flink state processor API. It works well when state types are the normal Java type or Java POJOs. When Avro generated Java classes are used as the state type, it does not read any states anymore. Are any additi

Write Kafka message header using FlinkKafkaProducer

Hi, I would like to add some meta data in the headers of kafka messages using FlinkKakfkaProducer. I googled a bit and have not found an example. Which Flink Serialization Schema should I use? Any suggestions? Thank you very much for your help in advance. Regards, Min E-mails can involve SUBS

RE: State Processor API and existing state

Hi, I have followed the steps below in restarting a Flink job with newly modified savepoints. I can re start a job with new savepoints as long as the Flink states are expressed in Java primitives. When the flink states are expressed in a POJO, my job does not get restarted. I have the followin

RE: State Processor API and existing state

. Regards, Min From: JING ZHANG Sent: 08 July 2021 04:45 To: Tan, Min Cc: Marco Villalobos ; user Subject: [External] Re: State Processor API and existing state Hi min, Is the POJO state in an existed operator or a new added operator? BTW, that would be great if you would like to give the code to

CoProcessFunction processElement1 and processElement2

Hi, Can someone confirm that the processElement1 and processElement2 are run sequentially? not in paradelle. Thank you very much in advance. Regards, Min E-mails can involve SUBSTANTIAL RISKS, e.g. lack of confidentiality, potential manipulation of contents and/or sender's address, incorre