OK. I think I should be able to get it in by 6pm PT, thanks to a quick +1 from Andrew, but certainly don't let it hold up the train if for some reason it takes longer than that.
-- Aaron T. Myers Software Engineer, Cloudera On Mon, Feb 10, 2014 at 12:04 PM, Arun C Murthy <a...@hortonworks.com> wrote: > Looks like we are down to 0 blockers; I'll create rc0 tonight. > > ATM - Your call, you have until 6pm tonight to get this in. > > thanks, > Arun > > On Feb 10, 2014, at 11:44 AM, "Aaron T. Myers" <a...@cloudera.com> wrote: > > > I just filed an issue for the fact that browsing the FS from the NN is > > broken if you have a directory with the sticky bit set: > > > > https://issues.apache.org/jira/browse/HDFS-5921 > > > > I didn't set this to be targeted for 2.3 because it doesn't seem like a > > _blocker_ to me, but if we're not going to get 2.3 out today anyway, I'd > > like to put this in. It's a small fix, and since many people have the > > sticky bit set on /tmp, they won't be able to browse any of the FS > > hierarchy from the NN without this fix. > > > > -- > > Aaron T. Myers > > Software Engineer, Cloudera > > > > > > On Fri, Feb 7, 2014 at 12:45 PM, Vinod Kumar Vavilapalli < > vino...@apache.org > >> wrote: > > > >> Heres what I've done: > >> - Reverted YARN-1493,YARN-1490,YARN-1041, > >> YARN-1166,YARN-1566,YARN-1689,YARN-1661 from branch-2.3. > >> - Updated YARN's CHANGES.txt in trunk, branch-2 and branch-2.3. > >> - Updated these JIRAs to have 2.4 as the fix-version. > >> - Compiled branch-2.3. > >> > >> Let me know if you run into any issues caused by this revert. > >> > >> Thanks, > >> +Vinod > >> > >> > >> On Fri, Feb 7, 2014 at 11:41 AM, Vinod Kumar Vavilapalli < > >> vino...@apache.org > >>> wrote: > >> > >>> Haven't heard back from Jian. Reverting the set from branch-2.3 (only). > >> Tx > >>> for the offline list. > >>> > >>> +Vinod > >>> > >>> > >>> On Fri, Feb 7, 2014 at 9:08 AM, Alejandro Abdelnur <t...@cloudera.com > >>> wrote: > >>> > >>>> Vinod, I have the patches to revert most of the JIRAs, the first > batch, > >>>> I'll send them off line to you. > >>>> > >>>> Thanks. > >>>> > >>>> > >>>> On Thu, Feb 6, 2014 at 8:56 PM, Vinod Kumar Vavilapalli > >>>> <vino...@apache.org>wrote: > >>>> > >>>>> > >>>>> Thanks. please post your findings, Jian wrote this part of the code > >> and > >>>>> between him/me, we can take care of those issues. > >>>>> > >>>>> +1 for going ahead with the revert on branch-2.3. I'll go do that > >>>> tomorrow > >>>>> morning unless I hear otherwise from Jian. > >>>>> > >>>>> Thanks, > >>>>> +Vinod > >>>>> > >>>>> > >>>>> On Feb 6, 2014, at 8:28 PM, Alejandro Abdelnur <t...@cloudera.com> > >>>> wrote: > >>>>> > >>>>>> Hi Vinod, > >>>>>> > >>>>>> Nothing confidential, > >>>>>> > >>>>>> * With umanaged AMs I'm seeing the trace I've posted a couple of > >> days > >>>> ago > >>>>>> in YARN-1577 ( > >>>>>> > >>>>> > >>>> > >> > https://issues.apache.org/jira/browse/YARN-1577?focusedCommentId=13891853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13891853 > >>>>>> ). > >>>>>> > >>>>>> * Also, Robert has been digging in Oozie testcases failing/getting > >>>> suck > >>>>>> with several token renewer threads, this failures happened > >>>> consistently > >>>>> at > >>>>>> different places around the same testcases (like some file > >> descriptors > >>>>>> leaking out), reverting YARN-1490 fixes the problem. The potential > >>>> issue > >>>>>> with this is that a long running client (oozie) my run into this > >>>>> situation > >>>>>> thus becoming unstable. > >>>>>> > >>>>>> *Robert,* mind posting to YARN-1490 the jvm thread dump at the time > >> of > >>>>> test > >>>>>> hanging? > >>>>>> > >>>>>> After YARN-1493 & YARN-1490 we have a couple of JIRAs trying to fix > >>>>> issues > >>>>>> introduced by them, and we still didn't get them right. > >>>>>> > >>>>>> Because this, the improvements driven by YARN-1493 & YARN-1490 seem > >>>> that > >>>>>> require more work before being stable. > >>>>>> > >>>>>> IMO, being conservative, we should do 2.3 without them and roll them > >>>> with > >>>>>> 2.4. If we want to do regular releases we will have to make this > >> kind > >>>> of > >>>>>> calls, else we will start dragging the releases. > >>>>>> > >>>>>> Sounds like a plan? > >>>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Thu, Feb 6, 2014 at 6:27 PM, Vinod Kumar Vavilapalli > >>>>>> <vino...@apache.org>wrote: > >>>>>> > >>>>>>> Hey > >>>>>>> > >>>>>>> I am not against removing them from 2.3 if that is helpful for > >>>> progress. > >>>>>>> But I want to understand what the issues are before we make that > >>>>> decision. > >>>>>>> > >>>>>>> There is the issue with unmanaged AM that is clearly known and I > >> was > >>>>>>> thinking of coming to the past two days, but couldn't. What is this > >>>> new > >>>>>>> issue that we (confidently?) pinned down to YARN-1490? > >>>>>>> > >>>>>>> Thanks > >>>>>>> +Vinod > >>>>>>> > >>>>>>> On Feb 6, 2014, at 5:07 PM, Alejandro Abdelnur <t...@cloudera.com> > >>>>> wrote: > >>>>>>> > >>>>>>>> Thanks Robert, > >>>>>>>> > >>>>>>>> All, > >>>>>>>> > >>>> > >>>>>>>> So it seems that YARN-1493 and YARN-1490 are introducing serious > >>>>>>>> regressions. > >>>>>>>> > >>>>>>>> I would propose to revert them and the follow up JIRAs from the > >> 2.3 > >>>>>>> branch > >>>>>>>> and keep working on them on trunk/branch-2 until the are stable (I > >>>>> would > >>>>>>>> even prefer reverting them from branch-2 not to block a 2.4 if > >> they > >>>> are > >>>>>>> not > >>>>>>>> ready in time). > >>>>>>>> > >>>>>>>> As I've mentioned before, the list of JIRAs to revert were: > >>>>>>>> > >>>>>>>> YARN-1493 > >>>>>>>> YARN-1490 > >>>>>>>> YARN-1166 > >>>>>>>> YARN-1041 > >>>>>>>> YARN-1566 > >>>>>>>> > >>>>>>>> Plus 2 additional JIRAs committed since my email on this issue 2 > >>>> days > >>>>>>> ago: > >>>>>>>> > >>>>>>>> *YARN-1661 > >>>>>>>> *YARN-1689 (not sure if this JIRA is related in functionality to > >> the > >>>>>>>> previous ones but it is creating conflicts). > >>>>>>>> > >>>>>>>> I think we should hold on continuing work on top of something that > >>>> is > >>>>>>>> broken until the broken stuff is fixed. > >>>>>>>> > >>>>>>>> Quoting Arun, "Committers - Henceforth, please use extreme caution > >>>>> while > >>>>>>>> committing to branch-2.3. Please commit *only* blockers to 2.3." > >>>>>>>> > >>>>>>>> YARN-1661 & YARN-1689 are not blockers. > >>>>>>>> > >>>>>>>> Unless there are objections, I'll revert all these JIRAs from > >>>>> branch-2.3 > >>>>>>>> tomorrow around noon and I'll update fixedVersion in the JIRAs. > >>>>>>>> > >>>>>>>> I'm inclined to revert them from branch-2 as well. > >>>>>>>> > >>>>>>>> Thoughts? > >>>>>>>> > >>>>>>>> Thanks. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, Feb 6, 2014 at 3:54 PM, Robert Kanter < > >> rkan...@cloudera.com > >>>>> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>>> I think we should revert YARN-1490 from Hadoop 2.3 branch. I > >>>> think it > >>>>>>> was > >>>>>>>>> causing some strange behavior in the Oozie unit tests: > >>>>>>>>> > >>>>>>>>> Basically, we use a single MiniMRCluster and MiniDFSCluster > >> across > >>>> all > >>>>>>> unit > >>>>>>>>> tests in a module. With YARN-1490 we saw that, regardless of > >> test > >>>>>>> order, > >>>>>>>>> the last few tests would timeout waiting for an MR job to finish; > >>>> on > >>>>>>> slower > >>>>>>>>> machines, the entire test suite would timeout. Through some > >>>> digging, > >>>>> I > >>>>>>>>> found that we were getting a ton of "Connection refused" > >>>> Exceptions on > >>>>>>>>> LeaseRenewer talking to the NN and a few on the AM talking to the > >>>> RM. > >>>>>>>>> > >>>>>>>>> After a bunch of investigation, I found that the problem went > >> away > >>>>> once > >>>>>>>>> YARN-1490 was removed. Though I couldn't figure out the exact > >>>>> problem. > >>>>>>>>> Even though this occurred in unit tests, it does make me > >> concerned > >>>>> that > >>>>>>> it > >>>>>>>>> could indicate some bigger issue in a long-running real cluster > >>>> (where > >>>>>>>>> everything isn't running on the same machine) that we haven't > >> seen > >>>>> yet. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Thu, Feb 6, 2014 at 3:06 PM, Karthik Kambatla < > >>>> ka...@cloudera.com> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I have marked MAPREDUCE-5744 a blocker for 2.3. Committing it > >>>>> shortly. > >>>>>>>>> Will > >>>>>>>>>> pull it out of branch-2.3 if anyone objects. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Thu, Feb 6, 2014 at 2:04 PM, Arpit Agarwal < > >>>>>>> aagar...@hortonworks.com > >>>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Merged HADOOP-10273 to branch-2.3 as r1565456. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Feb 5, 2014 at 4:49 PM, Arpit Agarwal < > >>>>>>>>> aagar...@hortonworks.com > >>>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> IMO HADOOP-10273 (Fix 'mvn site') should be included in 2.3. > >>>>>>>>>>>> > >>>>>>>>>>>> I will merge it to branch-2.3 tomorrow PST if no one > >> disagrees. > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Tue, Feb 4, 2014 at 5:03 PM, Alejandro Abdelnur < > >>>>>>>>> t...@cloudera.com > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> IMO YARN-1577 is a blocker, it is breaking unmanaged AMs in a > >>>> very > >>>>>>>>> odd > >>>>>>>>>>>>> ways > >>>>>>>>>>>>> (to the point it seems un-deterministic). > >>>>>>>>>>>>> > >>>>>>>>>>>>> I'd say eiher YARN-1577 is fixed or we revert > >>>>>>>>>>>>> YARN-1493/YARN-1490/YARN-1166/YARN-1041/YARN-1566 (almost > >> clean > >>>>>>>>>> reverts) > >>>>>>>>>>>>> from Hadoop 2.3 branch before doing the release. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> I've verified that after reverting those JIRAs things work > >> fine > >>>>> with > >>>>>>>>>>>>> unmanaged AMs. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Feb 4, 2014 at 11:45 AM, Arun C Murthy < > >>>>> a...@hortonworks.com > >>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> I punted YARN-1444 to 2.4 since it's a long-standing issue. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Jian is away and I don't see YARN-1577 & YARN-1206 making > >> much > >>>>>>>>>>> progress > >>>>>>>>>>>>>> till he is back; so I'm inclined to push both to 2.4 too. > >> Any > >>>>>>>>>>>>> objections? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Looks like Daryn has both HADOOP-10301 & HDFS-4564 covered. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Overall, I'll try get this out in next couple of days if we > >>>> can > >>>>>>>>>> clear > >>>>>>>>>>>>> the > >>>>>>>>>>>>>> list. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> thanks, > >>>>>>>>>>>>>> Arun > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Feb 3, 2014, at 12:14 PM, Arun C Murthy < > >>>> a...@hortonworks.com> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> An update. Per > https://s.apache.org/hadoop-2.3.0-blockerswe > >>>>>>>>> are > >>>>>>>>>>> now > >>>>>>>>>>>>>> down to 5 blockers: 1 Common, 1 HDFS, 3 YARN. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Daryn (thanks!) has both the non-YARN covered. Vinod is > >>>> helping > >>>>>>>>>> out > >>>>>>>>>>>>> with > >>>>>>>>>>>>>> the YARN ones. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> thanks, > >>>>>>>>>>>>>>> Arun > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> Arun C. Murthy > >>>>>>>>>>>>>> Hortonworks Inc. > >>>>>>>>>>>>>> http://hortonworks.com/ > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> CONFIDENTIALITY NOTICE > >>>>>>>>>>>>>> NOTICE: This message is intended for the use of the > >>>> individual or > >>>>>>>>>>>>> entity to > >>>>>>>>>>>>>> which it is addressed and may contain information that is > >>>>>>>>>>> confidential, > >>>>>>>>>>>>>> privileged and exempt from disclosure under applicable law. > >> If > >>>>> the > >>>>>>>>>>>>> reader > >>>>>>>>>>>>>> of this message is not the intended recipient, you are > >> hereby > >>>>>>>>>> notified > >>>>>>>>>>>>> that > >>>>>>>>>>>>>> any printing, copying, dissemination, distribution, > >>>> disclosure or > >>>>>>>>>>>>>> forwarding of this communication is strictly prohibited. If > >>>> you > >>>>>>>>> have > >>>>>>>>>>>>>> received this communication in error, please contact the > >>>> sender > >>>>>>>>>>>>> immediately > >>>>>>>>>>>>>> and delete it from your system. Thank You. > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> Alejandro > >>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> CONFIDENTIALITY NOTICE > >>>>>>>>>>> NOTICE: This message is intended for the use of the individual > >> or > >>>>>>>>> entity > >>>>>>>>>> to > >>>>>>>>>>> which it is addressed and may contain information that is > >>>>>>> confidential, > >>>>>>>>>>> privileged and exempt from disclosure under applicable law. If > >>>> the > >>>>>>>>> reader > >>>>>>>>>>> of this message is not the intended recipient, you are hereby > >>>>> notified > >>>>>>>>>> that > >>>>>>>>>>> any printing, copying, dissemination, distribution, disclosure > >> or > >>>>>>>>>>> forwarding of this communication is strictly prohibited. If you > >>>> have > >>>>>>>>>>> received this communication in error, please contact the sender > >>>>>>>>>> immediately > >>>>>>>>>>> and delete it from your system. Thank You. > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Alejandro > >>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> CONFIDENTIALITY NOTICE > >>>>>>> NOTICE: This message is intended for the use of the individual or > >>>>> entity to > >>>>>>> which it is addressed and may contain information that is > >>>> confidential, > >>>>>>> privileged and exempt from disclosure under applicable law. If the > >>>>> reader > >>>>>>> of this message is not the intended recipient, you are hereby > >>>> notified > >>>>> that > >>>>>>> any printing, copying, dissemination, distribution, disclosure or > >>>>>>> forwarding of this communication is strictly prohibited. If you > >> have > >>>>>>> received this communication in error, please contact the sender > >>>>> immediately > >>>>>>> and delete it from your system. Thank You. > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Alejandro > >>>> > >>>>> > >>>>> > >>>>> -- > >>>>> CONFIDENTIALITY NOTICE > >>>>> NOTICE: This message is intended for the use of the individual or > >>>> entity to > >>>>> which it is addressed and may contain information that is > >> confidential, > >>>>> privileged and exempt from disclosure under applicable law. If the > >>>> reader > >>>>> of this message is not the intended recipient, you are hereby > notified > >>>> that > >>>>> any printing, copying, dissemination, distribution, disclosure or > >>>>> forwarding of this communication is strictly prohibited. If you have > >>>>> received this communication in error, please contact the sender > >>>> immediately > >>>>> and delete it from your system. Thank You. > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Alejandro > >>>> > >>> > >>> > >> > >> -- > >> CONFIDENTIALITY NOTICE > >> NOTICE: This message is intended for the use of the individual or > entity to > >> which it is addressed and may contain information that is confidential, > >> privileged and exempt from disclosure under applicable law. If the > reader > >> of this message is not the intended recipient, you are hereby notified > that > >> any printing, copying, dissemination, distribution, disclosure or > >> forwarding of this communication is strictly prohibited. If you have > >> received this communication in error, please contact the sender > immediately > >> and delete it from your system. Thank You. > >> > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
