hi i am confused
the delay in in the source when reading message not on the sink nick בתאריך יום ב׳, 21 בדצמ׳ 2020 ב-18:12 מאת Yun Gao <yungao...@aliyun.com >: > Hi Nick, > > Are you using EXACTLY_ONCE semantics ? If so the sink would use > transactions, and only commit the transaction on checkpoint complete to > ensure end-to-end exactly-once. A detailed description could be find in [1] > > > Best, > Yun > > > [1] > https://flink.apache.org/features/2018/03/01/end-to-end-exactly-once-apache-flink.html > > ------------------------------------------------------------------ > Sender:nick toker<nick.toker....@gmail.com> > Date:2020/12/21 23:52:34 > Recipient:user<user@flink.apache.org> > Theme:checkpoint delay consume message > > Hello, > > We noticed the following behavior: > If we enable the flink checkpoints, we saw that there is a delay between > the time we write a message to the KAFKA topic and the time the flink kafka > connector consumes this message. > The delay is closely related to checkpointInterval and/or > minPauseBetweenCheckpoints meening that the MAX delay when consuming a > message from KAFKA will be one of these parameters. > > Could you please advise how we can remove/control this delay? > > we use flink 1.11.2 > > BR > nick > >
