Re: CHANGES.txt is gone from trunk, branch-2, branch-2.8

2016-03-08 Thread Colin P. McCabe
+1 Thanks, Andrew. This will avoid so many spurious conflicts when cherry-picking changes, and so much wasted time on commit. best, Colin On Thu, Mar 3, 2016 at 9:11 PM, Andrew Wang wrote: > Hi all, > > With the inclusion of HADOOP-12651 going back to branch-2.8, CHANGES.txt > and release note

Re: node.js and more as dependencies

2016-03-02 Thread Colin P. McCabe
mpilation/compaction code only exists in released code. I >> agree that we don't need obfuscation at all, but source code compaction >> could increase performance a lot, we could have heavy rendering tasks, such >> as visualization from statuses of 10K+ applications. Just like Java

Re: node.js and more as dependencies

2016-02-29 Thread Colin P. McCabe
Hmm. Devil's advocate here: Do we really need to have a "JS build"? The main use-cases for "JS builds" seem to be if you want to minimize or obfuscate your JS. Given that this is open source code, obfuscation seems unnecessary. Given that it's a low-traffic management interface, minimizing the

Re: Looking to a Hadoop 3 release

2016-02-22 Thread Colin P. McCabe
e released in 2.9. I think we > should rather concentrate our EC dev efforts to harden key features under > the follow-on umbrella HDFS-8031 and make it solid for a 3.0 release. > > Sincerely, > Zhe > > On Mon, Feb 22, 2016 at 9:25 AM Colin P. McCabe wrote: > >&g

Re: Looking to a Hadoop 3 release

2016-02-22 Thread Colin P. McCabe
+1 for a release of 3.0. There are a lot of significant, compatibility-breaking, but necessary changes in this release... we've touched on some of them in this thread. +1 for a parallel release of 2.8 as well. I think we are pretty close to this, barring a dozen or so blockers. best, Colin On

Re: Jenkins stability and patching

2015-11-23 Thread Colin P. McCabe
On Mon, Nov 23, 2015 at 1:53 PM, Colin P. McCabe wrote: > I agree that our tests are in a bad state. It would help if we could > maintain a list of "flaky tests" somewhere in git and have Yetus > consider the flakiness of a test before -1ing a patch. Right now, we > pre

Re: Jenkins stability and patching

2015-11-23 Thread Colin P. McCabe
I agree that our tests are in a bad state. It would help if we could maintain a list of "flaky tests" somewhere in git and have Yetus consider the flakiness of a test before -1ing a patch. Right now, we pretty much all have that list in our heads, and we're not applying it very consistently. Hav

Re: Java 8 + Jersey updates

2015-10-26 Thread Colin P. McCabe
Looks like a good idea. I assume you are targetting this only at trunk / 3.0 based on the "target version" and the incompatibility discussion? best, Colin On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa wrote: > Hi Steve, > > Thanks for your help. > > > 2. it's "significant" > > This change in

Re: [DISCUSS] Looking to a 2.8.0 release

2015-10-05 Thread Colin P. McCabe
I think it makes sense to have a 2.8 release since there are a tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x release seems like something we should consider separately since it would not have the same compatibility guarantees as a 2.8 release. There's a pretty big delta betwee

Re: Jenkins precommit-*-build

2015-05-05 Thread Colin P. McCabe
Thanks, Allen. This has long been a thorn in our side, and it's really good to see someone work on it. cheers, Colin On Tue, May 5, 2015 at 2:59 PM, Allen Wittenauer wrote: > TL;DR: > > Heads up: I’m going to hack on these scripts to fix the race > conditions. > > > > Pre

Re: [RESULT][VOTE] Release Apache Hadoop 2.7.0 RC0

2015-04-23 Thread Colin P. McCabe
Sorry for the late reply. It seems like the consensus is that we should push these fixes to 2.7.1. That works for me. HADOOP-11802 should be in there soon, hopefully the rest will follow quickly. best, Colin On Wed, Apr 22, 2015 at 4:27 PM, Vinod Kumar Vavilapalli wrote: > It took a while for

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

2015-04-17 Thread Colin P. McCabe
ption in line with >>my original proposal, following it up with a 2.7.1 in two weeks. >> >>Thanks >>+Vinod >> >>On Apr 17, 2015, at 2:27 AM, Colin P. McCabe wrote: >> >>> I would like to fix HDFS-8070, which just came to light. The impact >&

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

2015-04-16 Thread Colin P. McCabe
I would like to fix HDFS-8070, which just came to light. The impact is that if this isn't fixed, 2.6 clients will be unable to do short-circuit reads against 2.7 datanodes. best, Colin On Wed, Apr 15, 2015 at 8:19 PM, Brahma Reddy Battula wrote: > Need Jcardar changes to support java 7 Byte cod

Re: Hadoop - Major releases

2015-03-17 Thread Colin P. McCabe
Thanks, Andrew and Joep. +1 for maintaining wire and API compatibility, but moving to JDK8 in 3.0 best, Colin On Mon, Mar 16, 2015 at 3:22 PM, Andrew Wang wrote: > I took the liberty of adding line breaks to Joep's mail. > > Thanks for the great feedback Joep. The goal with 3.x is to maintain A

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

2015-03-10 Thread Colin P. McCabe
Er, that should read "as Allen commented" C. On Tue, Mar 10, 2015 at 11:55 AM, Colin P. McCabe wrote: > Hi Arun, > > Not all changes which are incompatible can be "fixed"-- sometimes an > incompatibility is a necessary part of a change. For example, taking &

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

2015-03-10 Thread Colin P. McCabe
way for the current > plan on hadoop-3.x right? So, I don't see the difference? > > Arun > > > From: Colin P. McCabe > Sent: Monday, March 09, 2015 3:05 PM > To: hdfs-...@hadoop.apache.org > Cc: mapreduce-...@hadoop.apache

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

2015-03-09 Thread Colin P. McCabe
Java 7 will be end-of-lifed in April 2015. I think it would be unwise to plan a new Hadoop release against a version of Java that is almost obsolete and (soon) no longer receiving security updates. I think people will be willing to roll out a new version of Java for Hadoop 3.x. Similarly, the wh

Re: DISCUSSION: Patch commit criteria.

2015-03-02 Thread Colin P. McCabe
I agree with Andrew and Konst here. I don't think the language is unclear in the rule, either... "consensus with a minimum of one +1" clearly indicates that _other people_ are involved, not just one person. I would also mention that we created the "branch committer" role specifically to make it e

Re: TimSort bug and its workaround

2015-03-02 Thread Colin P. McCabe
Thanks for bringing this up. If you can find any place where an array might realistically be larger than 67 million elements, then I guess file a JIRA for it. Also this array needs to be of objects, not of primitives (quicksort is used for those in jdk7, apparently). I can't think of any such pl

Re: Erratic Jenkins behavior

2015-02-18 Thread Colin P. McCabe
eed to download dependencies > fresh every time. > > Chris Nauroth > Hortonworks > http://hortonworks.com/ > > > > > > > On 2/12/15, 2:00 PM, "Colin P. McCabe" wrote: > >>We could potentially use different .m2 directories for each executor. >>I t

Re: Erratic Jenkins behavior

2015-02-12 Thread Colin P. McCabe
of our missing class error issues. Colin On Tue, Feb 10, 2015 at 2:13 AM, Steve Loughran wrote: > Mvn is a dark mystery to us all. I wouldn't trust it not pick up things from > other builds if they ended up published to ~/.m2/repository during the process > > > > On 9 Febr

Re: Erratic Jenkins behavior

2015-02-09 Thread Colin P. McCabe
I'm sorry, I don't have any insight into this. With regard to HADOOP-11084, I thought that $BUILD_URL would be unique for each concurrent build, which would prevent build artifacts from getting mixed up between jobs. Based on the value of PATCHPROCESS that Kihwal posted, perhaps this is not the c