回复:[DISCUSS] Releasing Flink 1.4
totally agree with the way.--发件人:Stephan Ewen 发送时间:2017年10月13日(星期五) 21:29收件人:dev@flink.apache.org 主 题:Re: [DISCUSS] Releasing Flink 1.4 I am in favor of doing this, if we can set it up in the following way. - We put out the 1.4 release now, as Till and Aljoscha suggested. A stable cut before the fundamental changes go in. - We merge the very big changes (FLIP-6, Network stack, localized state restore, etc). directly (or very soon) after. - We try to stabilize these changes and release 1.5 asap after that. Ideally Around end of year or so. The reason I am bringing this up is that I know various users waiting very much for FLIP-6 and Network Stack enhancements. Given that these issues were flagged for release 1.4, the users were planning to have them rather soon. Stephan On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek wrote: > +1 Excellent > > I'd like to volunteer as release manager. I already set up a Kanban board > to monitor the open blocking (and non-blocking) issues for 1.4, though this > is independent of me volunteering as release manager. We should all go over > these issues and see which ones should actually be blocking and which ones > are not yet on that list. > > > On 13. Oct 2017, at 12:24, Renjie Liu wrote: > > > > Cool!!! > > > > On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann > wrote: > > > >> Hi all, > >> > >> I want to revive the discussion about releasing Flink 1.4 [1] and the > set > >> of features to include. > >> > >> The gist of the previous discussion was that we postponed the feature > >> freeze for 1.4 in order to include some more features which were being > >> developed. By now, we have completed a good set of features such as > exactly > >> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and > >> many bug fixes. I believe that these features will make good release and > >> users are already waiting for them. > >> > >> Some of the other features which we wanted to include, mainly Flip-6, to > >> some extent the network stack enhancements and the state decoupling > still > >> need some more time. Since these features are major changes to Flink's > >> runtime, it would be in my opinion a good idea to cut a stable release > with > >> the above-mentioned feature set now and give the engine features a bit > more > >> time to ripen and be properly tested. > >> > >> Therefore, I would actually be in favour of aiming for a quick release > >> meaning that we now concentrate mainly on fixing bugs and critical > issues. > >> Moreover, I'm optimistic that the delayed features will be completed > soon > >> such that we can deliver them with the next release. What do you think? > >> > >> [1] > >> > >> http://apache-flink-mailing-list-archive.1008284.n3. > nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html > >> > >> Cheers, > >> Till > >> > > -- > > Liu, Renjie > > Software Engineer, MVAD > >
回复:[DISCUSS] Releasing Flink 1.4
@Stephan Ewen , do you think the barrier alignment improvement should be covered in release-1.4 or release-1.5?It can avoid spilling data on receiver side for exactly-once and speed the barrier alignment. We have implemented this feature based on new credit-based flow control and verified the good performance.If needed, I will try to submit the related PRs after network stack merged into 1.4 ASAP. --发件人:Aljoscha Krettek 发送时间:2017年10月18日(星期三) 03:04收件人:dev 主 题:Re: [DISCUSS] Releasing Flink 1.4 It seems we all agree on the general plan. Now we have to talk actual dates. I propose an aggressive feature-freeze date of October 31st (that would be in 2 weeks) so that we can release 1.4 early and can go full-speed with the features moved to 1.5. We can still merge in new features during the next two weeks but I propose that we then cut a release branch, start testing, and only merge bug fixes until the release. What do you think? > On 17. Oct 2017, at 08:37, Piotr Nowojski wrote: > > +1 > > I would only try to merge as many of the smaller network stack improvements >as possible for 1.4, since they give quite big performance improvement. > > Piotrek > >> On 16 Oct 2017, at 17:42, Eron Wright wrote: >> >> +1 from our side on this plan. >> >> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske wrote: >> >>> OK, sounds good to me. >>> >>> We have a couple of bugs to fix for the Table API / SQL but have PRs for >>> most of them. >>> >>> There's only one major issue that I'd like to include in 1.4.0 which is a >>> refactoring of the TableSource interface. >>> This effort has already started and is currently waiting for reviews / >>> comments. >>> I'm quite confident that we can get it in within the next two weeks. >>> >>> Cheers, Fabian >>> >>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek : >>> @Bowen I started marking essential stuff as blocking (with fixVersion 1.4.0). You're right, that we should start moving things to 1.5.0 that >>> are not blocking and that we don't think will make it into 1.4.0. I think we can only release 1.4.0 if there are 0 (zero) unresolved issues with fixVersion 1.4.0. > On 14. Oct 2017, at 07:34, Alexandru Gutan wrote: > > great > > On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) < > wangzhijiang...@aliyun.com> wrote: > >> totally agree with the way.-- >> 发件人:Stephan Ewen < se...@apache.org >>> 发送时间:2017年10月13日(星期五) 21:29收件人:dev@flink.apache.org < dev@flink.apache.org>主 >> 题:Re: [DISCUSS] Releasing Flink 1.4 >> I am in favor of doing this, if we can set it up in the following way. >> >> - We put out the 1.4 release now, as Till and Aljoscha suggested. A >> stable cut before the fundamental changes go in. >> >> - We merge the very big changes (FLIP-6, Network stack, localized >>> state >> restore, etc). directly (or very soon) after. >> - We try to stabilize these changes and release 1.5 asap after that. >> Ideally Around end of year or so. >> >> The reason I am bringing this up is that I know various users waiting very >> much for FLIP-6 and Network Stack enhancements. Given that these >>> issues >> were flagged for release 1.4, the users were planning to have them rather >> soon. >> >> Stephan >> >> >> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek < >>> aljos...@apache.org> >> wrote: >> >>> +1 Excellent >>> >>> I'd like to volunteer as release manager. I already set >> up a Kanban board >>> to monitor the open blocking (and non-blocking) issues >> for 1.4, though this >>> is independent of me volunteering as release manager. We >> should all go over >>> these issues and see which ones should actually be blockin >> g and which ones >>> are not yet on that list. >>> On 13. Oct 2017, at 12:24, Renjie Liu wrote: Cool!!! On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann >>> >>> wrote: > Hi all, > > I want to revive the discussion about releasing Flink 1.4 [1] and >>> the >>> set > of features to include. > > The gist of the previous discussion was that we postponed the >>> feature > freeze for 1.4 in order to include some more features w >> hich were being > developed. By now, we have completed a good set of features such as >>> exactly > once Kafka producer, reduced dependency footprint, Hado >> op-free Flink and > many bug fixes. I believe that these features will >> make good release and > users are already waiting for them. > > Some of the other features which we wanted to include, >> mainly Flip-6, t
回复:[DISCUSS] Releasing Flink 1.4
Thanks for replying @Till Rohrmann , and I agree with your suggestion. Cheers,Zhijiang --发件人:Till Rohrmann 发送时间:2017年10月18日(星期三) 16:27收件人:dev ; Zhijiang(wangzhijiang999) 抄 送:Stephan Ewen 主 题:Re: [DISCUSS] Releasing Flink 1.4 Thanks for driving this effort Aljoscha. I would be in favour of quick feature freeze (31st of October). However, looking at the list of open and in progress blockers we still have 53 blocker issues. I think it is unrealistic to complete all of them within the next two weeks. I would therefore propose to go again over the list of blockers to decide what is really a blocker for 1.4 and what not. @Zhijiang, my gut feeling is that we will have this feature in Flink 1.5. There are simply too many other open issues which we have to address first in order to finish the 1.4 release. Moreover, merging it early in the release cycle of 1.5 will give the barrier alignment improvements a good amount of exposure to spot potential problems before releasing it officially. Cheers, Till On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) < wangzhijiang...@aliyun.com> wrote: > @Stephan Ewen , do you think the barrier alignment improvement should be > covered in release-1.4 or release-1.5?It can avoid spilling data on > receiver side for exactly-once and speed the barrier alignment. > We have implemented this feature based on new credit-based flow control > and verified the good performance.If needed, I will try to submit the > related PRs after network stack merged into 1.4 ASAP. > --发件人:Aljoscha > Krettek 发送时间:2017年10月18日(星期三) 03:04收件人:dev < > dev@flink.apache.org>主 题:Re: [DISCUSS] Releasing Flink 1.4 > It seems we all agree on the general plan. Now we have to > talk actual dates. I propose an aggressive feature-freeze > date of October 31st (that would be in 2 weeks) so that > we can release 1.4 early and can go full-speed with the > features moved to 1.5. > > We can still merge in new features during the next two > weeks but I propose that we then cut a release branch, > start testing, and only merge bug fixes until the release. > What do you think? > > > On 17. Oct 2017, at 08:37, Piotr Nowojski > wrote: > > > > +1 > > > > I would only try to merge as many of the smaller > network stack improvements as possible for 1.4, since they > give quite big performance improvement. > > > > Piotrek > > > >> On 16 Oct 2017, at 17:42, Eron Wright wrote: > >> > >> +1 from our side on this plan. > >> > >> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske > wrote: > >> > >>> OK, sounds good to me. > >>> > >>> We have a couple of bugs to fix for the Table API / > SQL but have PRs for > >>> most of them. > >>> > >>> There's only one major issue that I'd like to > include in 1.4.0 which is a > >>> refactoring of the TableSource interface. > >>> This effort has already started and is currently waiting for reviews / > >>> comments. > >>> I'm quite confident that we can get it in within the next two weeks. > >>> > >>> Cheers, Fabian > >>> > >>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek : > >>> > @Bowen I started marking essential stuff as blocking (with fixVersion > 1.4.0). You're right, that we should start moving > things to 1.5.0 that > >>> are > not blocking and that we don't think will make it into > 1.4.0. I think we > can only release 1.4.0 if there are 0 (zero) unresolved issues with > fixVersion 1.4.0. > > > On 14. Oct 2017, at 07:34, Alexandru Gutan > wrote: > > > > great > > > > On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) < > > wangzhijiang...@aliyun.com> wrote: > > > >> totally agree with the way.-- > >> 发件人:Stephan Ewen < > se...@apache.org > >>> 发送时间:2017年10月13日(星期五) 21:29收件人:dev@flink.apache.org < > dev@flink.apache.org>主 > >> 题:Re: [DISCUSS] Releasing Flink 1.4 > >> I am in favor of doing this, if we can set it up in > the following way. > >> > >> - We put out the 1.4 release now, as Till and Aljoscha suggested. A > >> stable cut before the fundamental changes go in. > >> > >> - We merge the very big changes (FLIP-6, Network stack, localized > >>> state > >> restore, etc). directly (or very soon) after. > >> - We try to stabilize these changes and release 1.5 > asap after that. > >> Ideally Around end of year or so. > >> > >> The reason I am bringing this up is that I > know various users waiting > very > >> much for FLIP-6 and Network Stack enhancements. Given that these > >>> issues > >> were flagged for release 1.4, the users were planning to have them > rather > >> soon. > >> > >> Stephan > >> > >> > >> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek < > >>> aljos...@apache.org> > >>
回复:[DISCUSS] Releasing Flink 1.4
Hi Stephan! Thanks for replying. I am clearly now, all the big new features would be covered in 1.5. And it is very necessary to fully test big changes before release. Zhijiang --发件人:Stephan Ewen 发送时间:2017年10月18日(星期三) 22:54收件人:Till Rohrmann 抄 送:dev ; Zhijiang(wangzhijiang999) 主 题:Re: [DISCUSS] Releasing Flink 1.4 Hi Zhijiang! The proposal is to release 1.4 asap and have the master switch so 1.5 soon (early November or so). Merge the network code directly after that - the 1.5 release would come after flip-6, network stack, and local state caching is merged and tested. That gives us a better QA window. The reason is that we got feedback from Netflix and some other users that 1.3 had too many bugs, because we merged big changes directly before the release. We are worried that users lose trust in Flink of we release these big new features too fast and without intensively testing them. What do you think? Stephan On Oct 18, 2017 09:27, "Till Rohrmann" wrote: Thanks for driving this effort Aljoscha. I would be in favour of quick feature freeze (31st of October). However, looking at the list of open and in progress blockers we still have 53 blocker issues. I think it is unrealistic to complete all of them within the next two weeks. I would therefore propose to go again over the list of blockers to decide what is really a blocker for 1.4 and what not. @Zhijiang, my gut feeling is that we will have this feature in Flink 1.5. There are simply too many other open issues which we have to address first in order to finish the 1.4 release. Moreover, merging it early in the release cycle of 1.5 will give the barrier alignment improvements a good amount of exposure to spot potential problems before releasing it officially. Cheers,Till On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) wrote: @Stephan Ewen , do you think the barrier alignment improvement should be covered in release-1.4 or release-1.5?It can avoid spilling data on receiver side for exactly-once and speed the barrier alignment. We have implemented this feature based on new credit-based flow control and verified the good performance.If needed, I will try to submit the related PRs after network stack merged into 1.4 ASAP. --发件人:Aljoscha Krettek 发送时间:2017年10月18日(星期三) 03:04收件人:dev 主 题:Re: [DISCUSS] Releasing Flink 1.4 It seems we all agree on the general plan. Now we have to talk actual dates. I propose an aggressive feature-freeze date of October 31st (that would be in 2 weeks) so that we can release 1.4 early and can go full-speed with the features moved to 1.5. We can still merge in new features during the next two weeks but I propose that we then cut a release branch, start testing, and only merge bug fixes until the release. What do you think? > On 17. Oct 2017, at 08:37, Piotr Nowojski wrote: > > +1 > > I would only try to merge as many of the smaller network stack improvements >as possible for 1.4, since they give quite big performance improvement. > > Piotrek > >> On 16 Oct 2017, at 17:42, Eron Wright wrote: >> >> +1 from our side on this plan. >> >> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske wrote: >> >>> OK, sounds good to me. >>> >>> We have a couple of bugs to fix for the Table API / SQL but have PRs for >>> most of them. >>> >>> There's only one major issue that I'd like to include in 1.4.0 which is a >>> refactoring of the TableSource interface. >>> This effort has already started and is currently waiting for reviews / >>> comments. >>> I'm quite confident that we can get it in within the next two weeks. >>> >>> Cheers, Fabian >>> >>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek : >>> @Bowen I started marking essential stuff as blocking (with fixVersion 1.4.0). You're right, that we should start moving things to 1.5.0 that >>> are not blocking and that we don't think will make it into 1.4.0. I think we can only release 1.4.0 if there are 0 (zero) unresolved issues with fixVersion 1.4.0. > On 14. Oct 2017, at 07:34, Alexandru Gutan wrote: > > great > > On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) < > wangzhijiang...@aliyun.com> wrote: > >> totally agree with the way.-- >> 发件人:Stephan Ewen < se...@apache.org >>> 发送时间:2017年10月13日(星期五) 21:29收件人:dev@flink.apache.org < dev@flink.apache.org>主 >> 题:Re: [DISCUSS] Releasing Flink 1.4 >> I am in favor of doing this, if we can set it up in the following way. >> >> - We put out the 1.4 release now, as Till and Aljoscha suggested. A >> stable cut before the fundamental changes go in. >> >> - We merge the very big changes (FLIP-6, Network stack, localiz
Re: 回复:[DISCUSS] Releasing Flink 1.4
+1. To give users a heads up, might be good to start to change the ‘target release’ of some Jira tickets from 1.4 to 1.5. Sent from my iPhone > On Oct 13, 2017, at 11:02 PM, Zhijiang(wangzhijiang999) > wrote: > > totally agree with the > way.--发件人:Stephan > Ewen 发送时间:2017年10月13日(星期五) 21:29收件人:dev@flink.apache.org > 主 题:Re: [DISCUSS] Releasing Flink 1.4 > I am in favor of doing this, if we can set it up in the following way. > > - We put out the 1.4 release now, as Till and Aljoscha suggested. A > stable cut before the fundamental changes go in. > > - We merge the very big changes (FLIP-6, Network stack, localized state > restore, etc). directly (or very soon) after. > - We try to stabilize these changes and release 1.5 asap after that. > Ideally Around end of year or so. > > The reason I am bringing this up is that I know various users waiting very > much for FLIP-6 and Network Stack enhancements. Given that these issues > were flagged for release 1.4, the users were planning to have them rather > soon. > > Stephan > > > On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek > wrote: > >> +1 Excellent >> >> I'd like to volunteer as release manager. I already set up a Kanban board >> to monitor the open blocking (and non-blocking) issues for 1.4, though this >> is independent of me volunteering as release manager. We should all go over >> these issues and see which ones should actually be blocking and which ones >> are not yet on that list. >> >>> On 13. Oct 2017, at 12:24, Renjie Liu wrote: >>> >>> Cool!!! >>> On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann >>> wrote: >>> Hi all, I want to revive the discussion about releasing Flink 1.4 [1] and the >> set of features to include. The gist of the previous discussion was that we postponed the feature freeze for 1.4 in order to include some more features which were being developed. By now, we have completed a good set of features such as >> exactly once Kafka producer, reduced dependency footprint, Hadoop-free Flink and many bug fixes. I believe that these features will make good release and users are already waiting for them. Some of the other features which we wanted to include, mainly Flip-6, to some extent the network stack enhancements and the state decoupling >> still need some more time. Since these features are major changes to Flink's runtime, it would be in my opinion a good idea to cut a stable release >> with the above-mentioned feature set now and give the engine features a bit >> more time to ripen and be properly tested. Therefore, I would actually be in favour of aiming for a quick release meaning that we now concentrate mainly on fixing bugs and critical >> issues. Moreover, I'm optimistic that the delayed features will be completed >> soon such that we can deliver them with the next release. What do you think? [1] http://apache-flink-mailing-list-archive.1008284.n3. >> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html Cheers, Till >>> -- >>> Liu, Renjie >>> Software Engineer, MVAD >