I would consider these two blockers for 2.8.3 as they crash NN: https://issues.apache.org/jira/browse/HDFS-12638 https://issues.apache.org/jira/browse/HDFS-12832
Thanks, --Konstantin On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <j...@hortonworks.com> wrote: > Thanks Andrew and Wangda for comments! > > To me, an improvement with 17 patches is not a big problem as this is > self-contained and I didn't see a single line of delete/update on existing > code - well, arguably, patches with only adding code can also have big > impact but not likely the case here. > > While the dependency discussions on HADOOP-14964 are still going on, I > will leave the decision to JIRA discussion based on which approach we will > choose(shaded?) and impact. If we cannot make consensus in short term, > probably we have to miss this in 2.8.3 release. > > > Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0 > will get cut-off shortly. > > > > Thanks, > > > Junping > > > ________________________________ > From: Wangda Tan <wheele...@gmail.com> > Sent: Tuesday, November 21, 2017 10:52 AM > To: Andrew Wang > Cc: Junping Du; Zheng, Kai; common-...@hadoop.apache.org; > hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; > yarn-...@hadoop.apache.org > Subject: Re: Apache Hadoop 2.8.3 Release Plan > > Thanks Junping for driving this. > > For the bug fix vs. improvement, it is actually very hard to define, > improvement could be self-contained and useful, bug fix could be dangerous > in some cases. To me, If an improvement fixed some existing use case, and > the fix is self-contained. I will be open to bring such fix to maintenance > release. For example, in 2.8.2, we back ported CapacityScheduler intra > queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a > big change in terms of patch size, but since it fixes broken use case > (balance user usage under Capacity Scheduler leaf queue), we backported it > to 2.8.2 after thorough tests and validations by Yahoo. > > I'm not quite familiar with HADOOP-14964, I will leave the decision to > committers who know more about the field. > > Just my 2 cents. > > Regards, > Wangda > > > On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.w...@cloudera.com< > mailto:andrew.w...@cloudera.com>> wrote: > The Aliyun OSS code isn't a small improvement. If you look at Sammi's > comment > <https://issues.apache.org/jira/browse/HADOOP-14964? > focusedCommentId=16247085&page=com.atlassian.jira. > plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>, > it's > a 17 patch series that is being backported in one shot. What we're talking > about is equivalent to merging a feature branch in a maintenance release. I > see that Kai and Chris are having a discussion about the dependency > changes, which indicates this is not a zero-risk change either. We really > should not be changing dependency versions in a maintenance unless it's > because of a bug. > > It's unfortunate from a timing perspective that this missed 2.9.0, but I > still think it should wait for the next minor. Merging a feature into a > maintenance release sets the wrong precedent. > > Best, > Andrew > > On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <j...@hortonworks.com<mailto:jd > u...@hortonworks.com>> wrote: > > > Thanks Kai for calling out this feature/improvement for attention and > > Andrew for comments. > > > > > > While I agree that maintenance release should focus on important bug fix > > only, I doubt we have strict rules to disallow any features/improvements > to > > land on maint release especially when those are small footprint or low > > impact on existing code/features. In practice, we indeed has 77 new > > features/improvements in latest 2.7.3 and 2.7.4 release. > > > > > > Back to HADOOP-14964, I did a quick check and it looks like case here > > belongs to self-contained improvement that has very low impact on > existing > > code base, so I am OK with the improvement get landed on branch-2.8 in > case > > it is well reviewed and tested. > > > > > > However, as RM of branch-2.8, I have two concerns to accept it in our > > 2.8.3 release: > > > > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are > for > > several critical bug fixes and we should target to release it very soon - > > my current plan is to cut RC out within this week inline with waiting > > for 3.0.0 vote closing. Can this improvement be well tested against > > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we > > have strong commitment on test plan and activities in such a tight time. > > > > > > 2. Upgrading - I haven't heard we settle down the plan of releasing this > > feature in 2.9.1 release - though I saw some discussions are going on > > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes > > this improvement, then users consuming this feature/improvement have no > 2.9 > > release to upgrade or forcefully upgrade with regression. We may need a > > better upgrade story here. > > > > > > Pls let me know what you think. Thanks! > > > > > > > > Thanks, > > > > > > Junping > > > > > > ------------------------------ > > *From:* Andrew Wang <andrew.w...@cloudera.com<mailto: > andrew.w...@cloudera.com>> > > *Sent:* Monday, November 20, 2017 10:22 PM > > *To:* Zheng, Kai > > *Cc:* Junping Du; common-...@hadoop.apache.org<mailto:common-dev@hadoop. > apache.org>; hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org > >; > > mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>; > yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org> > > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan > > > > I'm against including new features in maintenance releases, since they're > > meant to be bug-fix only. > > > > If we're struggling with being able to deliver new features in a safe and > > timely fashion, let's try to address that, not overload the meaning of > > "maintenance release". > > > > Best, > > Andrew > > > > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zh...@intel.com<mailto: > kai.zh...@intel.com>> wrote: > > > >> Hi Junping, > >> > >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release. > >> > >> I have an ask, is it convenient to include the back port work for OSS > >> connector module? We have some Hadoop users that wish to have it by > default > >> for convenience, though in the past they used it by back porting > >> themselves. I have raised this and got thoughts from Chris and Steve. > Looks > >> like this is more wanted for 2.9 but I wanted to ask again here for > broad > >> feedback and thoughts by this chance. The back port patch is available > for > >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as > we > >> can see some shift from 2.7.x, hence it's worth more important features > and > >> efforts. How would you think? Thanks! > >> > >> https://issues.apache.org/jira/browse/HADOOP-14964 > >> > >> Regards, > >> Kai > >> > >> -----Original Message----- > >> From: Junping Du [mailto:j...@hortonworks.com<mailto:j...@hortonworks.com > >] > >> Sent: Tuesday, November 14, 2017 9:02 AM > >> To: common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>; > hdfs-...@hadoop.apache.org<mailto:hdfs-...@hadoop.apache.org>; > >> mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>; > yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org> > >> Subject: Apache Hadoop 2.8.3 Release Plan > >> > >> Hi, > >> We have several important fixes get landed on branch-2.8 and I would > >> like to cut off branch-2.8.3 now to start 2.8.3 release work. > >> So far, I don't see any pending blockers on 2.8.3, so my current > plan > >> is to cut off 1st RC of 2.8.3 in next several days: > >> - For all coming commits to land on branch-2.8, please mark > the > >> fix version as 2.8.4. > >> - If there is a really important fix for 2.8.3 and getting > >> closed, please notify me ahead before landing it on branch-2.8.3. > >> Please let me know if you have any thoughts or comments on the plan. > >> > >> Thanks, > >> > >> Junping > >> ________________________________________ > >> From: dujunp...@gmail.com<mailto:dujunp...@gmail.com> < > dujunp...@gmail.com<mailto:dujunp...@gmail.com>> on behalf of 俊平堵 < > >> junping...@apache.org<mailto:junping...@apache.org>> > >> Sent: Friday, October 27, 2017 3:33 PM > >> To: gene...@hadoop.apache.org<mailto:gene...@hadoop.apache.org> > >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release. > >> > >> Hi all, > >> > >> It gives me great pleasure to announce that the Apache Hadoop > >> community has voted to release Apache Hadoop 2.8.2, which is now > available > >> for download from Apache mirrors[1]. For download instructions please > refer > >> to the Apache Hadoop Release page [2]. > >> > >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line > and > >> our newest stable release for entire Apache Hadoop project. For major > >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main > page[3]. > >> > >> This release has 315 resolved issues since previous 2.8.1 release with > >> following > >> breakdown: > >> - 91 in Hadoop Common > >> - 99 in HDFS > >> - 105 in YARN > >> - 20 in MapReduce > >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details. > >> > >> The release news is posted on the Hadoop website too, you can go to the > >> downloads section directly [6]. > >> > >> Thank you all for contributing to the Apache Hadoop release! > >> > >> > >> Cheers, > >> > >> Junping > >> > >> > >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common > >> > >> [2] http://hadoop.apache.org/releases.html > >> > >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html > >> > >> [4] > >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had > >> oop-common/release/2.8.2/CHANGES.2.8.2.html > >> > >> [5] > >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had > >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html > >> > >> [6] http://hadoop.apache.org/releases.html#Download > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > <mailto:common-dev-unsubscr...@hadoop.apache.org> > >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org > <mailto:common-dev-h...@hadoop.apache.org> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org<mailto: > hdfs-dev-unsubscr...@hadoop.apache.org> > >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > <mailto:hdfs-dev-h...@hadoop.apache.org> > >> > >> > > > >
