Thanks for driving this, Xiangyu. We use Session clusters for quick SQL debugging internally, and found cold-start job submission slow due to lack of the exact minimum resource reservation feature proposed here. This should improve the experience a lot for running short lived-jobs in session clusters.
Best, Zhanghao Chen ________________________________ 发件人: Yangze Guo <karma...@gmail.com> 发送时间: 2023年9月19日 13:10 收件人: xiangyu feng <xiangyu...@gmail.com> 抄送: dev@flink.apache.org <dev@flink.apache.org> 主题: Re: [Discuss] FLIP-362: Support minimum resource limitation Thanks for driving this @Xiangyu. This is a feature that many users have requested for a long time. +1 for the overall proposal. Best, Yangze Guo On Tue, Sep 19, 2023 at 11:48 AM xiangyu feng <xiangyu...@gmail.com> wrote: > > Hi Devs, > > I'm opening this thread to discuss FLIP-362: Support minimum resource > limitation. The design doc can be found at: > FLIP-362: Support minimum resource limitation > > Currently, the Flink cluster only requests Task Managers (TMs) when there is > a resource requirement, and idle TMs are released after a certain period of > time. However, in certain scenarios, such as running short lived-jobs in > session cluster and scheduling batch jobs stage by stage, we need to improve > the efficiency of job execution by maintaining a certain number of available > workers in the cluster all the time. > > After discussed with Yangze, we introduced this new feature. The new added > public options and proposed changes are described in this FLIP. > > Looking forward to your feedback, thanks. > > Best regards, > Xiangyu >
