-1 (binding)
I feel like this FLIP needs a bit more time in the oven.
It seems to be very light on actual details; you can summarize the
entire changes section as "The enumerator calls this method and then
another checkpoint interval is used."
I would love to know how this is wired into the triggering of
checkpoints, what the behavior is with multiple sources, if a sink is
allowed to set this at any point or just once, what the semantics of a
"backlog" are for sources other than Hybrid/ MySQL CDC (because catching
up after a failover is a common enough pattern), whether/how this
information could also be interesting for the scheduler (because we may
want to avoid rescalings during the backlog processing), whether the
backlog processing should be exposed as a metric for users (or for that
matter, how we inform users at all that we're using a different
checkpoint interval at this time).
Following my discussion with Piotr and Stefan I'm also not sure how
future-proof the proposed API really is. Already I feel like the name
"setIsProcessingBacklog()" is rather specific for the state of the
source (making it technically wrong to call it in other situations like
being backpressured (again, depending on what "backlog processing" even
means)), while not being clear on what this actually results in. The
javadocs don't even mention the checkpointing interval at all, but
instead reference downstream optimizations that, afaict, aren't
mentioned in the FLIP.
I'd be very hesitant with marking it as public from the get-go. Ideally
it would maybe even be added as a separate interface (somehow).
On 30/06/2023 16:37, Piotr Nowojski wrote:
Hey,
Sorry to disturb this voting, but after discussing this thoroughly with
Chesnay and Stefan Richter I have to vote:
-1 (binding)
mainly to suspend the current voting thread. Please take a look at my mail
at dev mailing list.
Best,
Piotrek
czw., 29 cze 2023 o 14:59 feng xiangyu <xiangyu...@gmail.com> napisał(a):
+1 (non-binding)
Best,
Xiangyu
yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年6月29日周四 20:44写道：
+1 (binding)
Best regards,
Yuxia
----- 原始邮件 -----
发件人: "Yuepeng Pan" <flin...@126.com>
收件人: "dev" <dev@flink.apache.org>
发送时间: 星期四, 2023年 6 月 29日 下午 8:21:14
主题: Re: [VOTE] FLIP-309: Support using larger checkpointing interval when
source is processing backlog
+1 non-binding.
Best.
Yuepeng Pan
---- Replied Message ----
| From | Jingsong Li<jingsongl...@gmail.com> |
| Date | 06/29/2023 13:25 |
| To | dev<dev@flink.apache.org> |
| Cc | flink.zhouyunfeng<flink.zhouyunf...@gmail.com> |
| Subject | Re: [VOTE] FLIP-309: Support using larger checkpointing
interval when source is processing backlog |
+1 binding
On Thu, Jun 29, 2023 at 11:03 AM Dong Lin <lindon...@gmail.com> wrote:
Hi all,
We would like to start the vote for FLIP-309: Support using larger
checkpointing interval when source is processing backlog [1]. This FLIP
was
discussed in this thread [2].
Flink 1.18 release will feature freeze on July 11. We hope to make this
feature available in Flink 1.18.
The vote will be open until at least July 4th (at least 72 hours),
following
the consensus voting process.
Cheers,
Yunfeng and Dong
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-309%3A+Support+using+larger+checkpointing+interval+when+source+is+processing+backlog
[2] https://lists.apache.org/thread/l1l7f30h7zldjp6ow97y70dcthx7tl37
