Thank you for your answers.
We ended up changing the isolation level to read_uncommitted and it solves
the latency problem between the two jobs understanding that we may have
duplications in the second job when the first job fails and roll back.
בתאריך יום ג׳, 5 בינו׳ 2021 ב-15:23 מאת 赵一旦 :
> I
I think what you need is
http://kafka.apache.org/documentation/#consumerconfigs_isolation.level .
The isolation.level setting's default value is read_uncommitted. So, maybe
you do not use the default setting?
赵一旦 于2021年1月5日周二 下午9:10写道：
> I do not have this problem, so I guess it is related
I do not have this problem, so I guess it is related with the config of
your kafka producer and consumer, and maybe kafka topic properties or kafka
server properties also.
Arvid Heise 于2021年1月5日周二 下午6:47写道：
> Hi Daniel,
>
> Flink commits transactions on checkpoints while Kafka Streams/connect
>
Hi Daniel,
Flink commits transactions on checkpoints while Kafka Streams/connect
usually commits on record. This is the typical tradeoff between Throughput
and Latency. By decreasing the checkpoint interval in Flink, you can reach
comparable latency to Kafka Streams.
If you have two exactly once
I don't particularly know the our Kafka connector, but it sounds like a
matter of whether a given consumer does dirty reads.
Flink does not, whereas the other tools you're using do.
On 12/28/2020 7:57 AM, Daniel Peled wrote:
Hello,
We have 2 flink jobs that communicate with each other through
Hello,
We have 2 flink jobs that communicate with each other through a KAFKA topic.
Both jobs use checkpoints with EXACTLY ONCE semantic.
We have seen the following behaviour and we want to make sure and ask if
this is the expected behaviour or maybe it is a bug.
When the first job produces a
