Thanks Robert for all this, I think that we should also post a thread in the user ML so that users can also comment on the topic.
What do you think? Kostas On Mon, Mar 16, 2020 at 12:27 PM Robert Metzger <rmetz...@apache.org> wrote: > > Thank you all for your feedback. > > I will try to fix the test then (or disable it). > > Here's a ticket for dropping the BucketingSink: > https://issues.apache.org/jira/browse/FLINK-16616 Please mark whatever we > consider necessary as a "depends on" ticket. > @David / @Seth: Where are the tickets depending on FLIP-46 listed? Can one > of you add them to FLINK-16616 > > > On Fri, Mar 13, 2020 at 11:32 AM Guowei Ma <guowei....@gmail.com> wrote: > > > +1 to drop it. > > > > To Jingsong : > > we are planning to implement the orc StreamingFileSink in 1.11. > > I think users also could reference the old BucktSink from the old version. > > > > Best, > > Guowei > > > > > > Jingsong Li <jingsongl...@gmail.com> 于2020年3月13日周五 上午10:07写道： > > > > > Hi Robert, > > > > > > +1 to drop it but maybe not 1.11. > > > > > > ORC has not been supported on StreamingFileSink. I have seen lots of > > users > > > run ORC in the bucketing sink. > > > > > > Best, > > > Jingsong Lee > > > > > > On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman <sjwies...@gmail.com> > > wrote: > > > > > > > Sorry, I meant FLIP-46. > > > > > > > > Seth > > > > > > > > On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman <sjwies...@gmail.com> > > > wrote: > > > > > > > > > I agree with David, I think FLIP-49 needs to be prioritized for 1.11 > > if > > > > we > > > > > want to drop the bucketing sink. > > > > > > > > > > Seth > > > > > > > > > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson <da...@ververica.com > > > > > > > > wrote: > > > > > > > > > >> The BucketingSink is still somewhat widely used, I think in part > > > because > > > > >> of > > > > >> shortcomings in the StreamingFileSink. > > > > >> > > > > >> I would hope that in tandem with removing the bucketing sink we > > could > > > > also > > > > >> address some of these issues. I'm thinking in particular of issues > > > that > > > > >> are > > > > >> waiting on FLIP-46 [1]. > > > > >> > > > > >> Removing the bucketing sink will go down better, in my opinion, if > > > it's > > > > >> coupled with progress on some of the open StreamingFileSink tickets. > > > > >> > > > > >> Best, > > > > >> David > > > > >> > > > > >> [1] > > > > >> > > > > >> > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs > > > > >> > > > > >> > > > > >> On Thu, Mar 12, 2020 at 4:27 PM Zhijiang < > > wangzhijiang...@aliyun.com > > > > >> .invalid> > > > > >> wrote: > > > > >> > > > > >> > Thanks for driving this discussion, Robert! > > > > >> > > > > > >> > This e2e test really fails frequently. +1 to drop bucketing sink, > > > it > > > > is > > > > >> > not worth paying more efforts since deprecated. > > > > >> > > > > > >> > Best, > > > > >> > Zhijiang > > > > >> > > > > > >> > > > > > >> > ------------------------------------------------------------------ > > > > >> > From:Jeff Zhang <zjf...@gmail.com> > > > > >> > Send Time:2020 Mar. 12 (Thu.) 23:17 > > > > >> > To:dev <dev@flink.apache.org> > > > > >> > Subject:Re: [DISCUSS] Drop Bucketing Sink > > > > >> > > > > > >> > +1， dropping deprecated api is always necessary for a sustainable > > > > >> project. > > > > >> > > > > > >> > Kostas Kloudas <kklou...@gmail.com> 于2020年3月12日周四 下午11:06写道： > > > > >> > > > > > >> > > Hi Robert, > > > > >> > > > > > > >> > > +1 for dropping the BucketingSink. > > > > >> > > In any case, it has not been maintained for quite some time. > > > > >> > > > > > > >> > > Cheers, > > > > >> > > Kostas > > > > >> > > > > > > >> > > On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger < > > > rmetz...@apache.org > > > > > > > > > >> > > wrote: > > > > >> > > > > > > > >> > > > Hi all, > > > > >> > > > > > > > >> > > > I'm currently investigating a failing end to end test for the > > > > >> bucketing > > > > >> > > > sink [1]. > > > > >> > > > The bucketing sink has been deprecated in the 1.9 release [2], > > > > >> because > > > > >> > we > > > > >> > > > have the new StreamingFileSink [3] for quite a while. > > > > >> > > > Before putting any effort into fixing the end to end test for > > > the > > > > >> > sink, I > > > > >> > > > wanted to propose dropping the bucketing sink from master for > > > the > > > > >> > > upcoming > > > > >> > > > 1.11 release. > > > > >> > > > > > > > >> > > > What do you think? > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-16227 > > > > >> > > > [2] https://issues.apache.org/jira/browse/FLINK-13396 > > > > >> > > > [3] https://issues.apache.org/jira/browse/FLINK-9749 > > > > >> > > > > > > >> > > > > > >> > > > > > >> > -- > > > > >> > Best Regards > > > > >> > > > > > >> > Jeff Zhang > > > > >> > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > Best, Jingsong Lee > > > > >
