Hi Yun and Congxian, I would actually want checkpoint to avoid being triggered on a certain time. It still remains as system mechanism just avoid being triggered at a certain range of time. Waiting for the checkpoint to timeout still waste CPU&disk IO resources since it was being triggered. I would like it to avoid from being triggered at first. I suppose use a cron style would not break checkpoint's system mechanism. Savepoint, on the other hand, is not incremental update, trigger a savepoint every 10 mins will waste a lot of disk and another script is required to remove outdated savepoint. I suppose savepoint is being used in upgrade/restart scenario. A cron style checkpoint time config will provide a lot flexibility. Thanks.
On Thu, 21 Nov 2019 at 16:28, Yun Tang <myas...@live.com> wrote: > Hi Shuwen > > > > Conceptually, checkpoints in Flink behaves more like a system mechanism to > achieve fault tolerance and transparent for users. On the other hand, > savepoint in Flink behaves more like a user control behavior, can savepoint > not satisfy your demands for crontab? > > > > Best > > Yun Tang > > > > *From: *Congxian Qiu <qcx978132...@gmail.com> > *Date: *Thursday, November 21, 2019 at 2:27 PM > *To: *shuwen zhou <jaco...@gmail.com> > *Cc: *Jiayi Liao <bupt_...@163.com>, dev <d...@flink.apache.org>, user < > user@flink.apache.org> > *Subject: *Re: Cron style for checkpoint > > > > Hi > > > > Currently, Flink does not support such feature, from what you describe, > does set an appropriate timeout for checkpoint can solve your problem? > > > Best, > > Congxian > > > > > > shuwen zhou <jaco...@gmail.com> 于2019年11月21日周四 下午12:06写道： > > Hi Jiayi, > > It would be great if Flink could have a user defined interface for user to > implement to control checkpoint behavior, at least for time related > behavior. > > I brought up a wish on JIRA [1], perhaps it described clearly enough. > > > > [1] https://issues.apache.org/jira/browse/FLINK-14884 > > > > > > On Thu, 21 Nov 2019 at 11:40, Jiayi Liao <bupt_...@163.com> wrote: > > Hi Shuwen, > > > > As far as I know, Flink can only support checkpoint with a fixed interval. > > > > However I think the flexible mechanism of triggering checkpoint is worth > working on, at least from my perspective. And it may not only be a cron > style. In our business scenario, the data traffic usually reaches the peek > of the day after 20:00, which we want to increase the interval of > checkpoint otherwise it’ll introduce more disk and network IO. > > > > Just want to share something about this :) > > > > > > Best, > > Jiayi Liao > > > > > At 2019-11-21 10:20:47, "shuwen zhou" <jaco...@gmail.com> wrote: > > >Hi Community, > > >I would like to know if there is a existing function to support cron style > > >checkpoint? > > >The case is, our data traffic is huge on HH:30 for each hour. We don't wont > > >checkpoint to fall in that range of time. A cron like 15,45 * * * * to set > > >for checkpoint would be nice. If a checkpoint is already in progress when > > >minutes is 15 or 45, there would be a config value to trigger a new > > >checkpoint or pass. > > > > > >-- > > >Best Wishes, > > >Shuwen Zhou <http://www.linkedin.com/pub/shuwen-zhou/57/55b/599/> > > > > > > > > > > -- > > Best Wishes, > > Shuwen Zhou <http://www.linkedin.com/pub/shuwen-zhou/57/55b/599/> > > > > -- Best Wishes, Shuwen Zhou <http://www.linkedin.com/pub/shuwen-zhou/57/55b/599/>
