Re: Re-swizzle 2.3

2014-02-10 Thread Aaron T. Myers
There's still ongoing discussion on HDFS-4858 and I don't think we should hold up 2.3.0 for that. IMO we should target that for 2.3.1 or 2.4.0. -- Aaron T. Myers Software Engineer, Cloudera On Mon, Feb 10, 2014 at 5:53 PM, Konstantin Shvachko wrote: > Sorry for the last minute request. > Can we

Re: Re-swizzle 2.3

2014-02-10 Thread Konstantin Shvachko
Sorry for the last minute request. Can we add HDFS-4858 to the release, please? It solves pretty important bug related to failover. I can commit momentarily if there are no objections. Thanks, --Konstantin On Mon, Feb 10, 2014 at 4:50 PM, Aaron T. Myers wrote: > Just committed a fix for HDFS-5

Re: Re-swizzle 2.3

2014-02-10 Thread Aaron T. Myers
Just committed a fix for HDFS-5921 to branch-2.3. Fire away. -- Aaron T. Myers Software Engineer, Cloudera On Mon, Feb 10, 2014 at 1:34 PM, Aaron T. Myers wrote: > OK. I think I should be able to get it in by 6pm PT, thanks to a quick +1 > from Andrew, but certainly don't let it hold up the t

Re: Re-swizzle 2.3

2014-02-10 Thread Aaron T. Myers
OK. I think I should be able to get it in by 6pm PT, thanks to a quick +1 from Andrew, but certainly don't let it hold up the train if for some reason it takes longer than that. -- Aaron T. Myers Software Engineer, Cloudera On Mon, Feb 10, 2014 at 12:04 PM, Arun C Murthy wrote: > Looks like we

Re: Re-swizzle 2.3

2014-02-10 Thread Arun C Murthy
Looks like we are down to 0 blockers; I'll create rc0 tonight. ATM - Your call, you have until 6pm tonight to get this in. thanks, Arun On Feb 10, 2014, at 11:44 AM, "Aaron T. Myers" wrote: > I just filed an issue for the fact that browsing the FS from the NN is > broken if you have a directo

Re: Re-swizzle 2.3

2014-02-10 Thread Vinod Kumar Vavilapalli
I moved out HADOOP-10301 and HDFS-4564 out of 2.3 as they do not seem like regressions. Please revert back if you disagree. Given that, I think we are ready to cut a RC. +Vinod On Fri, Feb 7, 2014 at 12:45 PM, Vinod Kumar Vavilapalli wrote: > Heres what I've done: > - Reverted YARN-1493,YARN-

Re: Re-swizzle 2.3

2014-02-10 Thread Aaron T. Myers
I just filed an issue for the fact that browsing the FS from the NN is broken if you have a directory with the sticky bit set: https://issues.apache.org/jira/browse/HDFS-5921 I didn't set this to be targeted for 2.3 because it doesn't seem like a _blocker_ to me, but if we're not going to get 2.3

Re: Re-swizzle 2.3

2014-02-07 Thread Vinod Kumar Vavilapalli
Heres what I've done: - Reverted YARN-1493,YARN-1490,YARN-1041, YARN-1166,YARN-1566,YARN-1689,YARN-1661 from branch-2.3. - Updated YARN's CHANGES.txt in trunk, branch-2 and branch-2.3. - Updated these JIRAs to have 2.4 as the fix-version. - Compiled branch-2.3. Let me know if you run into any

Re: Re-swizzle 2.3

2014-02-07 Thread Vinod Kumar Vavilapalli
Haven't heard back from Jian. Reverting the set from branch-2.3 (only). Tx for the offline list. +Vinod On Fri, Feb 7, 2014 at 9:08 AM, Alejandro Abdelnur wrote: > Vinod, I have the patches to revert most of the JIRAs, the first batch, > I'll send them off line to you. > > Thanks. > > > On Thu,

Re: Re-swizzle 2.3

2014-02-07 Thread Alejandro Abdelnur
sire, as sandy said, lets keep it in branch 2 for now and if not resolved by 2.4 timeframe we'll revert them there. thx Alejandro (phone typing) > On Feb 7, 2014, at 10:14, Steve Loughran wrote: > >> On 6 February 2014 17:07, Alejandro Abdelnur wrote: >> >> Thanks Robert, >> >> All, >> >>

Re: Re-swizzle 2.3

2014-02-07 Thread Steve Loughran
On 6 February 2014 17:07, Alejandro Abdelnur wrote: > Thanks Robert, > > All, > > > > I'm inclined to revert them from branch-2 as well. > > -1 to that; if there are issues we should be able to find and fix them soon enough. Even if you aren't doing long-lived YARN services yet, even llama benefi

Re: Re-swizzle 2.3

2014-02-07 Thread Alejandro Abdelnur
Vinod, I have the patches to revert most of the JIRAs, the first batch, I'll send them off line to you. Thanks. On Thu, Feb 6, 2014 at 8:56 PM, Vinod Kumar Vavilapalli wrote: > > Thanks. please post your findings, Jian wrote this part of the code and > between him/me, we can take care of those

Re: Re-swizzle 2.3

2014-02-06 Thread Vinod Kumar Vavilapalli
Thanks. please post your findings, Jian wrote this part of the code and between him/me, we can take care of those issues. +1 for going ahead with the revert on branch-2.3. I'll go do that tomorrow morning unless I hear otherwise from Jian. Thanks, +Vinod On Feb 6, 2014, at 8:28 PM, Alejandro

Re: Re-swizzle 2.3

2014-02-06 Thread Alejandro Abdelnur
Hi Vinod, Nothing confidential, * With umanaged AMs I'm seeing the trace I've posted a couple of days ago in YARN-1577 ( https://issues.apache.org/jira/browse/YARN-1577?focusedCommentId=13891853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13891853 ). * Also, Rob

Re: Re-swizzle 2.3

2014-02-06 Thread Alejandro Abdelnur
yep, the idea is to pull all of them out from branch2.3. things go back to normal then. thanks Alejandro (phone typing) > On Feb 6, 2014, at 17:39, Zhijie Shen wrote: > > Recently I brought 4 JIRAs to branch-2.3, which are MAPREDUCE-5743, YARN-1628, > YARN-1661 and YARN-1689. Recall that we

Re: Re-swizzle 2.3

2014-02-06 Thread Sandy Ryza
+1 to reverting those JIRAs from branch-2.3. As YARN-1689 is fixing a problem caused by YARN-1493 I think we can revert it in branch-2.3 as well. I think we should leave them in branch-2 for now. We can revert if 2.4 is imminent and they're holding it up, but hopefully the issues they caused wil

Re: Re-swizzle 2.3

2014-02-06 Thread Vinod Kumar Vavilapalli
Hey I am not against removing them from 2.3 if that is helpful for progress. But I want to understand what the issues are before we make that decision. There is the issue with unmanaged AM that is clearly known and I was thinking of coming to the past two days, but couldn't. What is this new is

Re: Re-swizzle 2.3

2014-02-06 Thread Zhijie Shen
Recently I brought 4 JIRAs to branch-2.3, which are MAPREDUCE-5743, YARN-1628, YARN-1661 and YARN-1689. Recall that we mark test failure fixes as blockers for pior releases as closing to release, thus I brought to branch-2.3 MAPREDUCE-5743 and YARN-1628 that are the fixes for the test failure on 2.

Re: Re-swizzle 2.3

2014-02-06 Thread Alejandro Abdelnur
Thanks Robert, All, So it seems that YARN-1493 and YARN-1490 are introducing serious regressions. I would propose to revert them and the follow up JIRAs from the 2.3 branch and keep working on them on trunk/branch-2 until the are stable (I would even prefer reverting them from branch-2 not to bl

Re: Re-swizzle 2.3

2014-02-04 Thread Alejandro Abdelnur
IMO YARN-1577 is a blocker, it is breaking unmanaged AMs in a very odd ways (to the point it seems un-deterministic). I'd say eiher YARN-1577 is fixed or we revert YARN-1493/YARN-1490/YARN-1166/YARN-1041/YARN-1566 (almost clean reverts) from Hadoop 2.3 branch before doing the release. I've verif

Re: Re-swizzle 2.3

2014-02-04 Thread Arun C Murthy
I punted YARN-1444 to 2.4 since it's a long-standing issue. Jian is away and I don't see YARN-1577 & YARN-1206 making much progress till he is back; so I'm inclined to push both to 2.4 too. Any objections? Looks like Daryn has both HADOOP-10301 & HDFS-4564 covered. Overall, I'll try get this o

Re: Re-swizzle 2.3

2014-02-03 Thread Arun C Murthy
An update. Per https://s.apache.org/hadoop-2.3.0-blockers we are now down to 5 blockers: 1 Common, 1 HDFS, 3 YARN. Daryn (thanks!) has both the non-YARN covered. Vinod is helping out with the YARN ones. thanks, Arun -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of

Re: Re-swizzle 2.3

2014-02-01 Thread Vinod Kumar Vavilapalli
That's right. +Vinod On Jan 31, 2014, at 5:25 PM, Andrew Wang wrote: > YARN-1673 IIUC relates to the AHS, so is actually only in branch-2 and not > branch-2.3. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may

Re: Re-swizzle 2.3

2014-01-31 Thread Arun C Murthy
Thanks Vinod, appreciate it! I think we are very close. Here is a handy ref. to the list of blockers: http://s.apache.org/hadoop-2.3.0-blockers I'd appreciate if folks can help expedite these fixes, and, equally importantly bring up others they feel should be blockers for 2.3.0. thanks, Arun

Re: Re-swizzle 2.3

2014-01-30 Thread Vinod Kumar Vavilapalli
That was quite some exercise, but I'm done with it now. Updated YARN's and MAPREDUCE's CHANGES.txt on trunk, branch-2 and branch-2.3. Let me know if you find some inaccuracies. Thanks, +Vinod On Jan 29, 2014, at 10:49 PM, Vinod Kumar Vavilapalli wrote: > > Okay, I'll look at YARN and MR CHA

Re: Re-swizzle 2.3

2014-01-30 Thread Aaron T. Myers
I just committed HADOOP-10310 to branch-2.3, so we're good to go there. (Thanks to Andrew and Daryn for the prompt reviews.) -- Aaron T. Myers Software Engineer, Cloudera On Wed, Jan 29, 2014 at 6:52 PM, Aaron T. Myers wrote: > I just filed this JIRA as a blocker for 2.3: > https://issues.apac

Re: Re-swizzle 2.3

2014-01-29 Thread Vinod Kumar Vavilapalli
Okay, I'll look at YARN and MR CHANGES.txt problems. Seems like they aren't addressed yet. +Vinod On Jan 29, 2014, at 3:24 PM, Andrew Wang wrote: > I just finished tuning up branch-2.3 and fixing up the HDFS and Common > CHANGES.txt in trunk, branch-2, and branch-2.3. I had to merge back a f

Re: Re-swizzle 2.3

2014-01-29 Thread Stack
I filed https://issues.apache.org/jira/browse/HDFS-5852 as a blocker. See what ye all think. Thanks, St.Ack On Wed, Jan 29, 2014 at 3:52 PM, Aaron T. Myers wrote: > I just filed this JIRA as a blocker for 2.3: > https://issues.apache.org/jira/browse/HADOOP-10310 > > The tl;dr is that JNs will

Re: Re-swizzle 2.3

2014-01-29 Thread Aaron T. Myers
I just filed this JIRA as a blocker for 2.3: https://issues.apache.org/jira/browse/HADOOP-10310 The tl;dr is that JNs will not work with security enabled without this fix. If others don't think that supporting QJM with security enabled warrants a blocker for 2.3, then we can certainly lower the pr

Re: Re-swizzle 2.3

2014-01-29 Thread Doug Cutting
On Wed, Jan 29, 2014 at 12:30 PM, Jason Lowe wrote: > It is a bit concerning that the JIRA history showed that the target version > was set at some point in the past but no record of it being cleared. Perhaps the version itself was renamed? Doug

Re: Re-swizzle 2.3

2014-01-29 Thread Jason Lowe
I noticed that somehow the target version field in JIRA was invisibly cleared on most of the Blocker/Critical JIRAs that were originally targeted for 2.3.0/2.4.0. I happened to have an old browser tab lying around from an earlier query for these and I tried to fix them up, marking some for 2.4

Re: Re-swizzle 2.3

2014-01-29 Thread Arun C Murthy
Mostly ready for a jira perspective. Committers - Henceforth, please use extreme caution while committing to branch-2.3. Please commit *only* blockers to 2.3. thanks, Arun On Jan 28, 2014, at 3:30 PM, Arun C Murthy wrote: > Fixing up stuff now, thanks to Andrew for volunteering to help with

Re: Re-swizzle 2.3

2014-01-28 Thread Arun C Murthy
Fixing up stuff now, thanks to Andrew for volunteering to help with Common/HDFS. Arun On Jan 28, 2014, at 10:54 AM, Arun C Murthy wrote: > Sorry, missed this. Go ahead, I'll fix things up at the back end. Thanks. > > On Jan 28, 2014, at 12:11 AM, Sandy Ryza wrote: > >> Going forward with com

Re: Re-swizzle 2.3

2014-01-28 Thread Arun C Murthy
Sorry, missed this. Go ahead, I'll fix things up at the back end. Thanks. On Jan 28, 2014, at 12:11 AM, Sandy Ryza wrote: > Going forward with commits because it seems like others have been doing so > > > On Mon, Jan 27, 2014 at 1:31 PM, Sandy Ryza wrote: > >> We should hold off commits unti

Re: Re-swizzle 2.3

2014-01-28 Thread Sandy Ryza
Going forward with commits because it seems like others have been doing so On Mon, Jan 27, 2014 at 1:31 PM, Sandy Ryza wrote: > We should hold off commits until that's done, right? > > > On Mon, Jan 27, 2014 at 1:07 PM, Arun C Murthy wrote: > >> Yep, on it as we speak. :) >> >> >> Arun >> >> On

Re: Re-swizzle 2.3

2014-01-27 Thread Sandy Ryza
We should hold off commits until that's done, right? On Mon, Jan 27, 2014 at 1:07 PM, Arun C Murthy wrote: > Yep, on it as we speak. :) > > > Arun > > On Jan 27, 2014, at 12:36 PM, Jason Lowe wrote: > > > Thanks, Arun. Are there plans to update the Fix Versions and > CHANGES.txt accordingly?

Re: Re-swizzle 2.3

2014-01-27 Thread Arun C Murthy
Yep, on it as we speak. :) Arun On Jan 27, 2014, at 12:36 PM, Jason Lowe wrote: > Thanks, Arun. Are there plans to update the Fix Versions and CHANGES.txt > accordingly? There are a lot of JIRAs that are now going to ship in 2.3.0 > but the JIRA and CHANGES.txt says they're not fixed until

Re: Re-swizzle 2.3

2014-01-27 Thread Jason Lowe
Thanks, Arun. Are there plans to update the Fix Versions and CHANGES.txt accordingly? There are a lot of JIRAs that are now going to ship in 2.3.0 but the JIRA and CHANGES.txt says they're not fixed until 2.4.0. Jason On 01/27/2014 08:47 AM, Arun C Murthy wrote: Done. I've re-created branc

Re: Re-swizzle 2.3

2014-01-27 Thread Arun C Murthy
Done. I've re-created branch-2.3 from branch-2. thanks, Arun On Jan 23, 2014, at 2:40 AM, Arun Murthy wrote: > Based on the discussion at common-dev@, we've decided to target 2.3 > off the tip of branch-2 based on the 2 major HDFS features which are > Heterogenous Storage (HDFS-2832) and HDFS C

Re-swizzle 2.3

2014-01-23 Thread Arun Murthy
Based on the discussion at common-dev@, we've decided to target 2.3 off the tip of branch-2 based on the 2 major HDFS features which are Heterogenous Storage (HDFS-2832) and HDFS Cache (HDFS-4949). I'll create a new branch-2.3 on (1/24) at 6pm PST. thanks, Arun -- CONFIDENTIALITY NOTICE NOTICE: