Hi Chesnay, Thank you for your comments and I would be happy to discuss together to find a solution.
I just want to note that the discussion thread for this FLIP has been open for almost two months for everyone to leave comments. I will really appreciate it if in the future you can help provide comments earlier in the discussion thread so that I (and probably other contributors) can have the chance to address your concern and achieve consensus sooner than later. I am hoping we can be more considerate and help each other in the community be more productive. Thanks, Dong On Fri, Jun 30, 2023 at 11:18 PM Chesnay Schepler <ches...@apache.org> wrote: > -1 (binding) > > I feel like this FLIP needs a bit more time in the oven. > > It seems to be very light on actual details; you can summarize the > entire changes section as "The enumerator calls this method and then > another checkpoint interval is used." > I would love to know how this is wired into the triggering of > checkpoints, what the behavior is with multiple sources, if a sink is > allowed to set this at any point or just once, what the semantics of a > "backlog" are for sources other than Hybrid/ MySQL CDC (because catching > up after a failover is a common enough pattern), whether/how this > information could also be interesting for the scheduler (because we may > want to avoid rescalings during the backlog processing), whether the > backlog processing should be exposed as a metric for users (or for that > matter, how we inform users at all that we're using a different > checkpoint interval at this time). > > Following my discussion with Piotr and Stefan I'm also not sure how > future-proof the proposed API really is. Already I feel like the name > "setIsProcessingBacklog()" is rather specific for the state of the > source (making it technically wrong to call it in other situations like > being backpressured (again, depending on what "backlog processing" even > means)), while not being clear on what this actually results in. The > javadocs don't even mention the checkpointing interval at all, but > instead reference downstream optimizations that, afaict, aren't > mentioned in the FLIP. > > I'd be very hesitant with marking it as public from the get-go. Ideally > it would maybe even be added as a separate interface (somehow). > > On 30/06/2023 16:37, Piotr Nowojski wrote: > > Hey, > > > > Sorry to disturb this voting, but after discussing this thoroughly with > > Chesnay and Stefan Richter I have to vote: > > -1 (binding) > > mainly to suspend the current voting thread. Please take a look at my > mail > > at dev mailing list. > > > > Best, > > Piotrek > > > > czw., 29 cze 2023 o 14:59 feng xiangyu <xiangyu...@gmail.com> > napisał(a): > > > >> +1 (non-binding) > >> > >> Best, > >> Xiangyu > >> > >> yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年6月29日周四 20:44写道： > >> > >>> +1 (binding) > >>> > >>> Best regards, > >>> Yuxia > >>> > >>> ----- 原始邮件 ----- > >>> 发件人: "Yuepeng Pan" <flin...@126.com> > >>> 收件人: "dev" <dev@flink.apache.org> > >>> 发送时间: 星期四, 2023年 6 月 29日 下午 8:21:14 > >>> 主题: Re: [VOTE] FLIP-309: Support using larger checkpointing interval > when > >>> source is processing backlog > >>> > >>> +1 non-binding. > >>> > >>> > >>> Best. > >>> Yuepeng Pan > >>> > >>> > >>> ---- Replied Message ---- > >>> | From | Jingsong Li<jingsongl...@gmail.com> | > >>> | Date | 06/29/2023 13:25 | > >>> | To | dev<dev@flink.apache.org> | > >>> | Cc | flink.zhouyunfeng<flink.zhouyunf...@gmail.com> | > >>> | Subject | Re: [VOTE] FLIP-309: Support using larger checkpointing > >>> interval when source is processing backlog | > >>> +1 binding > >>> > >>> On Thu, Jun 29, 2023 at 11:03 AM Dong Lin <lindon...@gmail.com> wrote: > >>>> Hi all, > >>>> > >>>> We would like to start the vote for FLIP-309: Support using larger > >>>> checkpointing interval when source is processing backlog [1]. This > FLIP > >>> was > >>>> discussed in this thread [2]. > >>>> > >>>> Flink 1.18 release will feature freeze on July 11. We hope to make > this > >>>> feature available in Flink 1.18. > >>>> > >>>> The vote will be open until at least July 4th (at least 72 hours), > >>> following > >>>> the consensus voting process. > >>>> > >>>> Cheers, > >>>> Yunfeng and Dong > >>>> > >>>> [1] > >>>> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-309%3A+Support+using+larger+checkpointing+interval+when+source+is+processing+backlog > >>>> [2] https://lists.apache.org/thread/l1l7f30h7zldjp6ow97y70dcthx7tl37 > > >
