Thanks Junping for driving this. For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to bring such fix to maintenance release. For example, in 2.8.2, we back ported CapacityScheduler intra queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a big change in terms of patch size, but since it fixes broken use case (balance user usage under Capacity Scheduler leaf queue), we backported it to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to committers who know more about the field. Just my 2 cents. Regards, Wangda On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > The Aliyun OSS code isn't a small improvement. If you look at Sammi's > comment > <https://issues.apache.org/jira/browse/HADOOP-14964? > focusedCommentId=16247085&page=com.atlassian.jira. > plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>, > it's > a 17 patch series that is being backported in one shot. What we're talking > about is equivalent to merging a feature branch in a maintenance release. I > see that Kai and Chris are having a discussion about the dependency > changes, which indicates this is not a zero-risk change either. We really > should not be changing dependency versions in a maintenance unless it's > because of a bug. > > It's unfortunate from a timing perspective that this missed 2.9.0, but I > still think it should wait for the next minor. Merging a feature into a > maintenance release sets the wrong precedent. > > Best, > Andrew > > On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <j...@hortonworks.com> wrote: > > > Thanks Kai for calling out this feature/improvement for attention and > > Andrew for comments. > > > > > > While I agree that maintenance release should focus on important bug fix > > only, I doubt we have strict rules to disallow any features/improvements > to > > land on maint release especially when those are small footprint or low > > impact on existing code/features. In practice, we indeed has 77 new > > features/improvements in latest 2.7.3 and 2.7.4 release. > > > > > > Back to HADOOP-14964, I did a quick check and it looks like case here > > belongs to self-contained improvement that has very low impact on > existing > > code base, so I am OK with the improvement get landed on branch-2.8 in > case > > it is well reviewed and tested. > > > > > > However, as RM of branch-2.8, I have two concerns to accept it in our > > 2.8.3 release: > > > > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are > for > > several critical bug fixes and we should target to release it very soon - > > my current plan is to cut RC out within this week inline with waiting > > for 3.0.0 vote closing. Can this improvement be well tested against > > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we > > have strong commitment on test plan and activities in such a tight time. > > > > > > 2. Upgrading - I haven't heard we settle down the plan of releasing this > > feature in 2.9.1 release - though I saw some discussions are going on > > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes > > this improvement, then users consuming this feature/improvement have no > 2.9 > > release to upgrade or forcefully upgrade with regression. We may need a > > better upgrade story here. > > > > > > Pls let me know what you think. Thanks! > > > > > > > > Thanks, > > > > > > Junping > > > > > > ------------------------------ > > *From:* Andrew Wang <andrew.w...@cloudera.com> > > *Sent:* Monday, November 20, 2017 10:22 PM > > *To:* Zheng, Kai > > *Cc:* Junping Du; common-dev@hadoop.apache.org; > hdfs-...@hadoop.apache.org; > > mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org > > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan > > > > I'm against including new features in maintenance releases, since they're > > meant to be bug-fix only. > > > > If we're struggling with being able to deliver new features in a safe and > > timely fashion, let's try to address that, not overload the meaning of > > "maintenance release". > > > > Best, > > Andrew > > > > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zh...@intel.com> wrote: > > > >> Hi Junping, > >> > >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release. > >> > >> I have an ask, is it convenient to include the back port work for OSS > >> connector module? We have some Hadoop users that wish to have it by > default > >> for convenience, though in the past they used it by back porting > >> themselves. I have raised this and got thoughts from Chris and Steve. > Looks > >> like this is more wanted for 2.9 but I wanted to ask again here for > broad > >> feedback and thoughts by this chance. The back port patch is available > for > >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as > we > >> can see some shift from 2.7.x, hence it's worth more important features > and > >> efforts. How would you think? Thanks! > >> > >> https://issues.apache.org/jira/browse/HADOOP-14964 > >> > >> Regards, > >> Kai > >> > >> -----Original Message----- > >> From: Junping Du [mailto:j...@hortonworks.com] > >> Sent: Tuesday, November 14, 2017 9:02 AM > >> To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; > >> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org > >> Subject: Apache Hadoop 2.8.3 Release Plan > >> > >> Hi, > >> We have several important fixes get landed on branch-2.8 and I would > >> like to cut off branch-2.8.3 now to start 2.8.3 release work. > >> So far, I don't see any pending blockers on 2.8.3, so my current > plan > >> is to cut off 1st RC of 2.8.3 in next several days: > >> - For all coming commits to land on branch-2.8, please mark > the > >> fix version as 2.8.4. > >> - If there is a really important fix for 2.8.3 and getting > >> closed, please notify me ahead before landing it on branch-2.8.3. > >> Please let me know if you have any thoughts or comments on the plan. > >> > >> Thanks, > >> > >> Junping > >> ________________________________________ > >> From: dujunp...@gmail.com <dujunp...@gmail.com> on behalf of 俊平堵 < > >> junping...@apache.org> > >> Sent: Friday, October 27, 2017 3:33 PM > >> To: gene...@hadoop.apache.org > >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release. > >> > >> Hi all, > >> > >> It gives me great pleasure to announce that the Apache Hadoop > >> community has voted to release Apache Hadoop 2.8.2, which is now > available > >> for download from Apache mirrors[1]. For download instructions please > refer > >> to the Apache Hadoop Release page [2]. > >> > >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line > and > >> our newest stable release for entire Apache Hadoop project. For major > >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main > page[3]. > >> > >> This release has 315 resolved issues since previous 2.8.1 release with > >> following > >> breakdown: > >> - 91 in Hadoop Common > >> - 99 in HDFS > >> - 105 in YARN > >> - 20 in MapReduce > >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details. > >> > >> The release news is posted on the Hadoop website too, you can go to the > >> downloads section directly [6]. > >> > >> Thank you all for contributing to the Apache Hadoop release! > >> > >> > >> Cheers, > >> > >> Junping > >> > >> > >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common > >> > >> [2] http://hadoop.apache.org/releases.html > >> > >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html > >> > >> [4] > >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had > >> oop-common/release/2.8.2/CHANGES.2.8.2.html > >> > >> [5] > >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had > >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html > >> > >> [6] http://hadoop.apache.org/releases.html#Download > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >> > >> > > >
