Hi Xuannan,
I like that you are cleaning up options that I assume are not recommended or
currently used in some way.
I have not got experience of these options. For the proposed deprecations, will
there be a proposed recommended alternatives that will be mentioned in the
deprecation. If they are going to be removed for v2, it would be good to
explicitly document the thinking and impact of the deprecations and consequent
removals,
Kind regards, David.
From: Xuannan Su <suxuanna...@gmail.com>
Date: Tuesday, 14 May 2024 at 02:08
To: dev@flink.apache.org <dev@flink.apache.org>
Subject: [EXTERNAL] Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration
for Flink 2.0
Hi all,
Thank you for all the comments and suggestions! If there are no
further comments, I'd like to close the discussion and start the
voting in two days.
Best regards,
Xuannan
On Mon, May 13, 2024 at 3:10 PM Jeyhun Karimov <je.kari...@gmail.com> wrote:
>
> Hi Xuannan,
>
> Thanks a lot for the update. The FLIP looks good to me. +1 for it.
>
> Regards,
> Jeyhun
>
> On Mon, May 13, 2024 at 4:45 AM Xuannan Su <suxuanna...@gmail.com> wrote:
>
> > Hi Jeyhun,
> >
> > Thanks for the comment!
> >
> > Yes, we intended to remove the StreamPiplineOptions in 2.0. I updated
> > the FLIP to include the information.
> >
> > Best regards,
> > Xuannan
> >
> > On Sun, May 12, 2024 at 9:16 PM Jeyhun Karimov <je.kari...@gmail.com>
> > wrote:
> > >
> > > Hi Xuannan,
> > >
> > > Thanks for driving this FLIP!
> > > I have a minor comment. Do we plan to remove StreamPipelineOptions in
> > 2.0,
> > > as it only contains deprecated options?
> > >
> > > Regards,
> > > Jeyhun
> > >
> > > On Sat, May 11, 2024 at 4:40 AM Rui Fan <1996fan...@gmail.com> wrote:
> > >
> > > > Thanks Xuannan for the update!
> > > >
> > > > LGTM, +1 for this proposal.
> > > >
> > > > Best,
> > > > Rui
> > > >
> > > > On Sat, May 11, 2024 at 10:20 AM Xuannan Su <suxuanna...@gmail.com>
> > wrote:
> > > >
> > > > > Hi Rui,
> > > > >
> > > > > Thanks for the suggestion!
> > > > >
> > > > > I updated the description of
> > > > > taskmanager.network.memory.max-overdraft-buffers-per-gate and
> > > > > hard-coded it to 20.
> > > > >
> > > > > Best regards,
> > > > > Xuannan
> > > > >
> > > > > On Mon, May 6, 2024 at 11:28 AM Rui Fan <1996fan...@gmail.com>
> > wrote:
> > > > > >
> > > > > > Thanks Xuannan for driving this proposal!
> > > > > >
> > > > > > > taskmanager.network.memory.max-overdraft-buffers-per-gate will be
> > > > > removed
> > > > > > and hard-coded to either 10 or 20.
> > > > > >
> > > > > > Currently, it's a public option. Could we determine the value of
> > > > > > the overdraft buffer in the current FLIP?
> > > > > >
> > > > > > I vote 20 as the hard code value due to 2 reasons:
> > > > > > - Removing this option means users cannot change it, it might be
> > better
> > > > > to
> > > > > > turn it up.
> > > > > > - Most of tasks don't use the overdraft buffer, so increasing it
> > > > doesn't
> > > > > > introduce more risk.
> > > > > >
> > > > > > Best,
> > > > > > Rui
> > > > > >
> > > > > > On Mon, May 6, 2024 at 10:47 AM Yuxin Tan <tanyuxinw...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Thanks for the effort, Xuannan.
> > > > > > >
> > > > > > > +1 for the proposal.
> > > > > > >
> > > > > > > Best,
> > > > > > > Yuxin
> > > > > > >
> > > > > > >
> > > > > > > Xintong Song <tonysong...@gmail.com> 于2024年4月29日周一 15:40写道：
> > > > > > >
> > > > > > > > Thanks for driving this effort, Xuannan.
> > > > > > > >
> > > > > > > > +1 for the proposed changes.
> > > > > > > >
> > > > > > > > Just one suggestion: Some of the proposed changes involve not
> > > > solely
> > > > > > > > changing the configuration options, but are bound to changing /
> > > > > removal
> > > > > > > of
> > > > > > > > certain features. E.g., the removal of hash-blocking shuffle
> > and
> > > > > legacy
> > > > > > > > hybrid shuffle mode, and the behavior change of overdraft
> > network
> > > > > > > buffers.
> > > > > > > > Therefore, it might be nicer to provide an implementation plan
> > > > with a
> > > > > > > list
> > > > > > > > of related tasks in the FLIP. This should not block the FLIP
> > > > though.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > >
> > > > > > > > Xintong
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Apr 25, 2024 at 4:35 PM Xuannan Su <
> > suxuanna...@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > I'd like to start a discussion on FLIP-450: Improve Runtime
> > > > > > > > > Configuration for Flink 2.0 [1]. As Flink moves toward 2.0,
> > we
> > > > have
> > > > > > > > > revisited all runtime configurations and identified several
> > > > > > > > > improvements to enhance user-friendliness and
> > maintainability. In
> > > > > this
> > > > > > > > > FLIP, we aim to refine the runtime configuration.
> > > > > > > > >
> > > > > > > > > Looking forward to everyone's feedback and suggestions. Thank
> > > > you!
> > > > > > > > >
> > > > > > > > > Best regards,
> > > > > > > > > Xuannan
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-450%3A+Improve+Runtime+Configuration+for+Flink+2.0
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
Unless otherwise stated above:
IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
