Re: [VOTE] Release Apache Hadoop 2.1.0-beta
With 9 +1s including mine (6 binding) and no -1s, the vote passes. Thanks to all who voted. Arun On Aug 15, 2013, at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1, binding Review process # symlink /usr/local/bin/protoc to the homebrew installed 2.5.0 version # delete all 2.1.0-beta artifacts in the mvn repo: find ~/.m2 -name 2.1.0-beta -print | xargs rm -rf # checkout hbase apache/branch-0.95 (commit # b58d596 ) # switch to ASF repo (arun's private repo is in the POM, with JARs with the same sha1 sum, I'm just being rigorous) ASF Staging https://repository.apache.org/content/groups/staging/ # clean build of hbase tar against the beta artifacts mvn clean install assembly:single -DskipTests -Dmaven.javadoc.skip=true -Dhadoop.profile=2.0 -Dhadoop-two.version=2.1.0-beta # Observe DL taking place ] [INFO] --- maven-assembly-plugin:2.4:single (default-cli) @ hbase --- [INFO] Assemblies have been skipped per configuration of the skipAssembly parameter. [INFO] [INFO] [INFO] Building HBase - Common 0.95.3-SNAPSHOT [INFO] Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-annotations/2.1.0-beta/hadoop-annotations-2.1.0-beta.pom Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-annotations/2.1.0-beta/hadoop-annotations-2.1.0-beta.pom(2 KB at 3.3 KB/sec) Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-project/2.1.0-beta/hadoop-project-2.1.0-beta.pom Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-project/2.1.0-beta/hadoop-project-2.1.0-beta.pom(34 KB at 383.8 KB/sec) Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-main/2.1.0-beta/hadoop-main-2.1.0-beta.pom Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-main/2.1.0-beta/hadoop-main-2.1.0-beta.pom(17 KB at 184.0 KB/sec) Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.pom Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.pom(26 KB at 293.3 KB/sec) Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-project-dist/2.1.0-beta/hadoop-project-dist-2.1.0-beta.pom Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-project-dist/2.1.0-beta/hadoop-project-dist-2.1.0-beta.pom(17 KB at 176.2 KB/sec) Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-auth/2.1.0-beta/hadoop-auth-2.1.0-beta.pom Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-auth/2.1.0-beta/hadoop-auth-2.1.0-beta.pom(7 KB at 79.0 KB/sec) Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-annotations/2.1.0-beta/hadoop-annotations-2.1.0-beta.jar Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.jar Downloading: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-auth/2.1.0-beta/hadoop-auth-2.1.0-beta.jar Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-annotations/2.1.0-beta/hadoop-annotations-2.1.0-beta.jar(17 KB at 146.1 KB/sec) Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-auth/2.1.0-beta/hadoop-auth-2.1.0-beta.jar(47 KB at 241.7 KB/sec) Downloaded: https://repository.apache.org/content/groups/staging/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.jar(2657 KB at 2260.9 KB/sec) [INFO] # get md5 sum of hadoop-common-2.1.0-beta artifact in https://repository.apache.org/content/groups/staging/ : 0166f5c94d3699b3a37efc16ebb1ceea3acb3b53 # verify version of artifact in local m2 repo $ sha1sum ~/.m2/repository/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.jar 0166f5c94d3699b3a37efc16ebb1ceea3acb3b53 # in hbase/hbase-assembly/target , gunzip then untar the hbase-0.95.3-SNAPSHOT-bin.tar file # Patch the Hoya POM to use 2.1.0-beta instead of a local 2.1.1-SNAPSHOT # run some of the hbase cluster deploy & flexing tests mvn clean test -Pstaging (all tests pass after 20 min) == functional tests = # build and the Hoya JAR with classpath mvn package -Pstaging # D/L the binary .tar.gz file, and scp to an ubuntu VM with the hadoop conf properties for net-accessible HDFS & YARN services, no memory limits on containers https://github.com/hortonworks/hoya/tree/master/src/test/configs/ubuntu https://github.com/hortonworks/hoya/blob/master/src/test/configs/ubuntu/core-site.xml https://github.com/hortonworks/hoya/blob/master/src/test/configs/ubuntu/yarn-site.xml # stop the running hadoop-2.1.1-snapshot cluster # start the new cluster serv
RE: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (non-binding) Bikas -Original Message- From: Hitesh Shah [mailto:hit...@apache.org] Sent: Wednesday, August 21, 2013 4:25 PM To: yarn-...@hadoop.apache.org; Arun Murthy Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta +1 (binding) Verified checksums, built from source and ran basic MR jobs on a single-node cluster. -- Hitesh On Aug 15, 2013, at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta- > rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or > entity to which it is addressed and may contain information that is > confidential, privileged and exempt from disclosure under applicable > law. If the reader of this message is not the intended recipient, you > are hereby notified that any printing, copying, dissemination, > distribution, disclosure or forwarding of this communication is > strictly prohibited. If you have received this communication in error, > please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 non-binding Verified signatures and checksums one binary and source tar files Built the source Ran some tests on psuedo-distributed cluster. Rob On 08/15/2013 09:15 PM, Arun C Murthy wrote: Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Wed, Aug 21, 2013 at 3:49 PM, Stack wrote: > On Wed, Aug 21, 2013 at 1:25 PM, Colin McCabe wrote: > >> St.Ack wrote: >> >> > + Once I figured where the logs were, found that JAVA_HOME was not being >> > exported (don't need this in hadoop-2.0.5 for instance). Adding an >> > exported JAVA_HOME to my running shell which don't seem right but it took >> > care of it (I gave up pretty quick on messing w/ >> > yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't >> > getting anywhere) >> >> I thought that we were always supposed to have JAVA_HOME set when >> running any of these commands. At least, I do. How else can the >> system disambiguate between different Java installs? I need 2 >> installs to test with JDK7. >> >> > > That is fair enough but I did not need to define this explicitly previously > (for hadoop-2.0.5-alpha for instance) or the JAVA_HOME that was figured in > start scripts was propagated and now is not (I have not dug in). > > > >> > + This did not seem to work for me: >> > hadoop.security.group.mapping >> > >> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback> > lue>. >> >> We've seen this before. I think your problem is that you have >> java.library.path set correctly (what System.loadLibrary checks), but >> your system library path does not include a necessary dependency of >> libhadoop.so-- most likely, libjvm.so. Probably, we should fix >> NativeCodeLoader to actually make a function call in libhadoop.so >> before it declares everything OK. >> > > My expectation was that if native group lookup fails, as it does here, then > the 'Fallback' would kick in and we'd do the Shell query. This mechanism > does not seem to be working. I filed https://issues.apache.org/jira/browse/HADOOP-9895 to address this issue. best, Colin > > > St.Ack
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (binding) Verified checksums, built from source and ran basic MR jobs on a single-node cluster. -- Hitesh On Aug 15, 2013, at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (binding) - verified signatures and checksums - built from source - ran some simple jobs on a single-node cluster On 08/15/2013 04:15 PM, Arun C Murthy wrote: Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Wed, Aug 21, 2013 at 1:25 PM, Colin McCabe wrote: > St.Ack wrote: > > > + Once I figured where the logs were, found that JAVA_HOME was not being > > exported (don't need this in hadoop-2.0.5 for instance). Adding an > > exported JAVA_HOME to my running shell which don't seem right but it took > > care of it (I gave up pretty quick on messing w/ > > yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't > > getting anywhere) > > I thought that we were always supposed to have JAVA_HOME set when > running any of these commands. At least, I do. How else can the > system disambiguate between different Java installs? I need 2 > installs to test with JDK7. > > That is fair enough but I did not need to define this explicitly previously (for hadoop-2.0.5-alpha for instance) or the JAVA_HOME that was figured in start scripts was propagated and now is not (I have not dug in). > > + This did not seem to work for me: > > hadoop.security.group.mapping > > > org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback > lue>. > > We've seen this before. I think your problem is that you have > java.library.path set correctly (what System.loadLibrary checks), but > your system library path does not include a necessary dependency of > libhadoop.so-- most likely, libjvm.so. Probably, we should fix > NativeCodeLoader to actually make a function call in libhadoop.so > before it declares everything OK. > My expectation was that if native group lookup fails, as it does here, then the 'Fallback' would kick in and we'd do the Shell query. This mechanism does not seem to be working. St.Ack
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
St.Ack wrote: > + Once I figured where the logs were, found that JAVA_HOME was not being > exported (don't need this in hadoop-2.0.5 for instance). Adding an > exported JAVA_HOME to my running shell which don't seem right but it took > care of it (I gave up pretty quick on messing w/ > yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't > getting anywhere) I thought that we were always supposed to have JAVA_HOME set when running any of these commands. At least, I do. How else can the system disambiguate between different Java installs? I need 2 installs to test with JDK7. > + This did not seem to work for me: > hadoop.security.group.mapping > org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback lue>. We've seen this before. I think your problem is that you have java.library.path set correctly (what System.loadLibrary checks), but your system library path does not include a necessary dependency of libhadoop.so-- most likely, libjvm.so. Probably, we should fix NativeCodeLoader to actually make a function call in libhadoop.so before it declares everything OK. Colin On Tue, Aug 20, 2013 at 5:35 PM, Stack wrote: > On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > >> Folks, >> >> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >> like to get released - this fixes the bugs we saw since the last go-around >> (rc1). >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> > > It basically works (in insecure mode), +1. > > + Checked signature. > + Ran on small cluster w/ small load made using mapreduce interfaces. > + Got the HBase full unit test suite to pass on top of it. > > I had the following issues getting it to all work. I don't know if they are > known issues so will just list them here first. > > + I could not find documentation on how to go from tarball to running > cluster (the bundled 'cluster' and 'standalone' doc are not about how to > get this tarball off the ground). > + I had a bit of a struggle putting this release in place under hbase unit > tests. The container would just exit w/ 127 errcode. No logs in expected > place. Tripped over where minimrcluster was actually writing. Tried to > corral it so it played nicely w/o our general test setup but found that the > new mini clusters have 'target' hardcoded as output dirs. > + Once I figured where the logs were, found that JAVA_HOME was not being > exported (don't need this in hadoop-2.0.5 for instance). Adding an > exported JAVA_HOME to my running shell which don't seem right but it took > care of it (I gave up pretty quick on messing w/ > yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't > getting anywhere) > + This did not seem to work for me: > hadoop.security.group.mapping > org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback. > It just did this: > > Caused by: java.lang.UnsatisfiedLinkError: > org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V > at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native > Method) > at > org.apache.hadoop.security.JniBasedUnixGroupsMapping.(JniBasedUnixGroupsMapping.java:49) > at > org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.(JniBasedUnixGroupsMappingWithFallback.java:38) > > ..so I replaced it > w/ org.apache.hadoop.security.ShellBasedUnixGroupsMapping on the hbase-side > to get my cluster up and running. > > + Untarring the bin dir, it undoes as hadoop-X.Y.Z-beta. Undoing the src > dir it undoes as hadoop-X.Y.Z-beta-src. I'd have thought they would undo > into the one directory overlaying each other. > > St.Ack
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (binding). Verified signature and checksums. Built from source and upgraded a 2.0.4 cluster. Ran a couple of simple MR and Tez jobs. - Sid On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (non-binding) Built from source, ran jobs on a pseudo-distributed cluster with the Fair Scheduler. -Sandy On Tue, Aug 20, 2013 at 7:27 PM, Arun C Murthy wrote: > Thanks for the heads up Aaron, I've changed fix-version of HDFS-4763 to > 2.1.1-beta for now. > > Committers - please be careful setting fix-versions, this is a good > anti-pattern to avoid… though, I'm willing to bet a lot of dough that this > isn't the first Hadoop release with this issue… *smile* > > Arun > > > On Aug 20, 2013, at 6:09 PM, Aaron T. Myers wrote: > > > I was evaluating the release bits when I noticed that the change done in > > HDFS-4763 to add support for starting the HDFS NFSv3 gateway, which is > > marked with a "fix version" of 2.1.0-beta and included in the release > notes > > of RC2, is not in fact included in the RC2 release bits. It looks to me > > like the change is included in branch-2.1-beta, but not > branch-2.1.0-beta. > > > > Particularly since the release notes in RC2 are incorrect in claiming > that > > this change is in this release, it seems like a pretty serious > > issue. Ordinarily I'd say that this issue should result in a new RC, and > I > > would vote -1 on RC2. But, given the previous discussion that folks are > > interested in releasing 2.1.0-beta with several fairly substantial bugs > > that we already know about, I'll withhold my vote. If RC2 ends up getting > > released as-is, we should be sure to change the fix version field on that > > JIRA to be correct. > > > > -- > > Aaron T. Myers > > Software Engineer, Cloudera > > > > > > On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy > wrote: > > > >> Folks, > >> > >> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I > would > >> like to get released - this fixes the bugs we saw since the last > go-around > >> (rc1). > >> > >> The RC is available at: > >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > >> The RC tag in svn is here: > >> > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > >> > >> The maven artifacts are available via repository.apache.org. > >> > >> Please try the release and vote; the vote will run for the usual 7 days. > >> > >> thanks, > >> Arun > >> > >> -- > >> Arun C. Murthy > >> Hortonworks Inc. > >> http://hortonworks.com/ > >> > >> > >> > >> -- > >> CONFIDENTIALITY NOTICE > >> NOTICE: This message is intended for the use of the individual or > entity to > >> which it is addressed and may contain information that is confidential, > >> privileged and exempt from disclosure under applicable law. If the > reader > >> of this message is not the intended recipient, you are hereby notified > that > >> any printing, copying, dissemination, distribution, disclosure or > >> forwarding of this communication is strictly prohibited. If you have > >> received this communication in error, please contact the sender > immediately > >> and delete it from your system. Thank You. > >> > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Thanks for the heads up Aaron, I've changed fix-version of HDFS-4763 to 2.1.1-beta for now. Committers - please be careful setting fix-versions, this is a good anti-pattern to avoid… though, I'm willing to bet a lot of dough that this isn't the first Hadoop release with this issue… *smile* Arun On Aug 20, 2013, at 6:09 PM, Aaron T. Myers wrote: > I was evaluating the release bits when I noticed that the change done in > HDFS-4763 to add support for starting the HDFS NFSv3 gateway, which is > marked with a "fix version" of 2.1.0-beta and included in the release notes > of RC2, is not in fact included in the RC2 release bits. It looks to me > like the change is included in branch-2.1-beta, but not branch-2.1.0-beta. > > Particularly since the release notes in RC2 are incorrect in claiming that > this change is in this release, it seems like a pretty serious > issue. Ordinarily I'd say that this issue should result in a new RC, and I > would vote -1 on RC2. But, given the previous discussion that folks are > interested in releasing 2.1.0-beta with several fairly substantial bugs > that we already know about, I'll withhold my vote. If RC2 ends up getting > released as-is, we should be sure to change the fix version field on that > JIRA to be correct. > > -- > Aaron T. Myers > Software Engineer, Cloudera > > > On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > >> Folks, >> >> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >> like to get released - this fixes the bugs we saw since the last go-around >> (rc1). >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> >> thanks, >> Arun >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I was evaluating the release bits when I noticed that the change done in HDFS-4763 to add support for starting the HDFS NFSv3 gateway, which is marked with a "fix version" of 2.1.0-beta and included in the release notes of RC2, is not in fact included in the RC2 release bits. It looks to me like the change is included in branch-2.1-beta, but not branch-2.1.0-beta. Particularly since the release notes in RC2 are incorrect in claiming that this change is in this release, it seems like a pretty serious issue. Ordinarily I'd say that this issue should result in a new RC, and I would vote -1 on RC2. But, given the previous discussion that folks are interested in releasing 2.1.0-beta with several fairly substantial bugs that we already know about, I'll withhold my vote. If RC2 ends up getting released as-is, we should be sure to change the fix version field on that JIRA to be correct. -- Aaron T. Myers Software Engineer, Cloudera On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > It basically works (in insecure mode), +1. + Checked signature. + Ran on small cluster w/ small load made using mapreduce interfaces. + Got the HBase full unit test suite to pass on top of it. I had the following issues getting it to all work. I don't know if they are known issues so will just list them here first. + I could not find documentation on how to go from tarball to running cluster (the bundled 'cluster' and 'standalone' doc are not about how to get this tarball off the ground). + I had a bit of a struggle putting this release in place under hbase unit tests. The container would just exit w/ 127 errcode. No logs in expected place. Tripped over where minimrcluster was actually writing. Tried to corral it so it played nicely w/o our general test setup but found that the new mini clusters have 'target' hardcoded as output dirs. + Once I figured where the logs were, found that JAVA_HOME was not being exported (don't need this in hadoop-2.0.5 for instance). Adding an exported JAVA_HOME to my running shell which don't seem right but it took care of it (I gave up pretty quick on messing w/ yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't getting anywhere) + This did not seem to work for me: hadoop.security.group.mapping org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback. It just did this: Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.(JniBasedUnixGroupsMappingWithFallback.java:38) ..so I replaced it w/ org.apache.hadoop.security.ShellBasedUnixGroupsMapping on the hbase-side to get my cluster up and running. + Untarring the bin dir, it undoes as hadoop-X.Y.Z-beta. Undoing the src dir it undoes as hadoop-X.Y.Z-beta-src. I'd have thought they would undo into the one directory overlaying each other. St.Ack
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Mon, Aug 19, 2013 at 2:49 PM, Vinod Kumar Vavilapalli wrote: > > Thanks for the clarification Daryn. That is what I was asking before when I > said "From my limited understanding, this doesn't seem like a API or a > compatibility issue. Can we not fix it in subsequent bug-fix releases? " > > Now, I was wavering as to whether we can omit this or not. There are other > such things like YARN-1082 (RM restart doesn't work in security), YARN-49 > (dist-shell in secure mode) etc. that is essentially a loss of some > functionality in some cases (secure mode). > > Flipping back the question, aren't we okay release now with all the API > changes that are already in and then immediately follow up with a bunch of > bug-fix releases or wait till 'everything' gets fixed. > > Posing the question like that, I am willing to move ahead without waiting > for these fixes, but that's just me. > > What do others think? Pushing a 2.1.0 out this week and following up with 2.1.1 in Sep might actually be a pretty good plan. That said, we have to be *really* conservative on what goes into 2.1.1 and make sure we can have it out on time. Thanks, Roman.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Vinod, your thinking makes sense to me. My two cents are that we should hold off on fixes until 2.1.1-beta. Unless there are downstream projects that need it to work for integration testing. -Sandy On Mon, Aug 19, 2013 at 2:49 PM, Vinod Kumar Vavilapalli wrote: > > Thanks for the clarification Daryn. That is what I was asking before when > I said "From my limited understanding, this doesn't seem like a API or a > compatibility issue. Can we not fix it in subsequent bug-fix releases? " > > Now, I was wavering as to whether we can omit this or not. There are other > such things like YARN-1082 (RM restart doesn't work in security), YARN-49 > (dist-shell in secure mode) etc. that is essentially a loss of some > functionality in some cases (secure mode). > > Flipping back the question, aren't we okay release now with all the API > changes that are already in and then immediately follow up with a bunch of > bug-fix releases or wait till 'everything' gets fixed. > > Posing the question like that, I am willing to move ahead without waiting > for these fixes, but that's just me. > > What do others think? > > Thanks, > +Vinod > > On Aug 19, 2013, at 8:25 AM, Daryn Sharp wrote: > > I've been OOO (got a call to fix this bug), but just to clarify: > > We're ok with HA _not working at all_ with security enabled in 2.1.0-beta? > That's the ramification of omitting HADOOP-9880. > > Daryn > > On Aug 16, 2013, at 9:20 PM, Arun C Murthy wrote: > > Yep, there are quite a number of such fixes in 2.1.1 ATM, I think it will > serve us better to get 2.1.0 out and then quickly turn around to make 2.1.1. > > > My current plan is to start work on 2.1.1 right after this release gets > complete… hopefully next week. > > > thanks, > > Arun > > > On Aug 16, 2013, at 4:36 PM, Vinod Kumar Vavilapalli < > vino...@hortonworks.com> wrote: > > > > There are other such isolated and well understood bug-fixes that we pushed > to 2.1.1 in the interesting of making progress with 2.1.0 and the > corresponding API changes. > > > 2.1.1 should happen soon enough after this. > > > Thanks, > > +Vinod > > > On Aug 16, 2013, at 4:22 PM, Roman Shaposhnik wrote: > > > What are the downsides of getting this fix into the 2.1? It appears > > that the fix is pretty isolated and well understood. > > > Thoughts? > > > Thanks, > > Roman. > > > On Fri, Aug 16, 2013 at 3:04 PM, Kihwal Lee wrote: > > I've changed the target version of HADOOP-9880 to 2.1.1. Please change it > back, if you feel that it needs to be in 2.1.0-beta. > > > > Kihwal > > > > ____ > > From: Kihwal Lee > > To: Arun Murthy ; "common-dev@hadoop.apache.org" < > common-dev@hadoop.apache.org> > > Cc: "mapreduce-...@hadoop.apache.org" ; " > hdfs-...@hadoop.apache.org" ; " > yarn-...@hadoop.apache.org" > > Sent: Friday, August 16, 2013 4:55 PM > > Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta > > > > It's your call, Arun. I.e. as long you believe rc2 meets the expectations > and objectives of 2.1.0-beta. > > > Kihwal > > > > > > From: Arun Murthy > > To: "common-dev@hadoop.apache.org" > > Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" < > mapreduce-...@hadoop.apache.org>; "hdfs-...@hadoop.apache.org" < > hdfs-...@hadoop.apache.org>; "yarn-...@hadoop.apache.org" < > yarn-...@hadoop.apache.org> > > Sent: Friday, August 16, 2013 3:44 PM > > Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta > > > > That makes sense too. > > > > On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli > > wrote: > > > > We need to make a call on what blockers will be. From my limited > understanding, this doesn't seem like a API or a compatibility issue. Can > we not fix it in subsequent bug-fix releases? > > > I do see a lot of follow up releases to 2.1.0. Getting this release out > will help downstream projects start testing with all the API stuff that has > already gone in 2.1.0. > > > Thanks, > > +Vinod > > > On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: > > > We have found HADOOP-9880, which prevents Namenode HA from running with > security. > > > Kihwal > > > > > > From: Arun C Murthy > > To: "common-dev@hadoop.apache.org" ; " > hdfs-...@hadoop.apache.org" ; " > mapred
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Thanks for the clarification Daryn. That is what I was asking before when I said "From my limited understanding, this doesn't seem like a API or a compatibility issue. Can we not fix it in subsequent bug-fix releases? " Now, I was wavering as to whether we can omit this or not. There are other such things like YARN-1082 (RM restart doesn't work in security), YARN-49 (dist-shell in secure mode) etc. that is essentially a loss of some functionality in some cases (secure mode). Flipping back the question, aren't we okay release now with all the API changes that are already in and then immediately follow up with a bunch of bug-fix releases or wait till 'everything' gets fixed. Posing the question like that, I am willing to move ahead without waiting for these fixes, but that's just me. What do others think? Thanks, +Vinod On Aug 19, 2013, at 8:25 AM, Daryn Sharp wrote: > I've been OOO (got a call to fix this bug), but just to clarify: > > We're ok with HA _not working at all_ with security enabled in 2.1.0-beta? > That's the ramification of omitting HADOOP-9880. > > Daryn > > On Aug 16, 2013, at 9:20 PM, Arun C Murthy wrote: > >> Yep, there are quite a number of such fixes in 2.1.1 ATM, I think it will >> serve us better to get 2.1.0 out and then quickly turn around to make 2.1.1. >> >> My current plan is to start work on 2.1.1 right after this release gets >> complete… hopefully next week. >> >> thanks, >> Arun >> >> On Aug 16, 2013, at 4:36 PM, Vinod Kumar Vavilapalli >> wrote: >> >>> >>> There are other such isolated and well understood bug-fixes that we pushed >>> to 2.1.1 in the interesting of making progress with 2.1.0 and the >>> corresponding API changes. >>> >>> 2.1.1 should happen soon enough after this. >>> >>> Thanks, >>> +Vinod >>> >>> On Aug 16, 2013, at 4:22 PM, Roman Shaposhnik wrote: >>> >>>> What are the downsides of getting this fix into the 2.1? It appears >>>> that the fix is pretty isolated and well understood. >>>> >>>> Thoughts? >>>> >>>> Thanks, >>>> Roman. >>>> >>>> On Fri, Aug 16, 2013 at 3:04 PM, Kihwal Lee wrote: >>>>> I've changed the target version of HADOOP-9880 to 2.1.1. Please change >>>>> it back, if you feel that it needs to be in 2.1.0-beta. >>>>> >>>>> >>>>> Kihwal >>>>> >>>>> >>>>> >>>>> From: Kihwal Lee >>>>> To: Arun Murthy ; "common-dev@hadoop.apache.org" >>>>> >>>>> Cc: "mapreduce-...@hadoop.apache.org" ; >>>>> "hdfs-...@hadoop.apache.org" ; >>>>> "yarn-...@hadoop.apache.org" >>>>> Sent: Friday, August 16, 2013 4:55 PM >>>>> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >>>>> >>>>> >>>>> It's your call, Arun. I.e. as long you believe rc2 meets the >>>>> expectations and objectives of 2.1.0-beta. >>>>> >>>>> Kihwal >>>>> >>>>> >>>>> >>>>> From: Arun Murthy >>>>> To: "common-dev@hadoop.apache.org" >>>>> Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" >>>>> ; "hdfs-...@hadoop.apache.org" >>>>> ; "yarn-...@hadoop.apache.org" >>>>> >>>>> Sent: Friday, August 16, 2013 3:44 PM >>>>> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >>>>> >>>>> >>>>> That makes sense too. >>>>> >>>>> >>>>> On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli >>>>> wrote: >>>>> >>>>>> >>>>>> We need to make a call on what blockers will be. From my limited >>>>>> understanding, this doesn't seem like a API or a compatibility issue. >>>>>> Can we not fix it in subsequent bug-fix releases? >>>>>> >>>>>> I do see a lot of follow up releases to 2.1.0. Getting this release out >>>>>> will help downstream projects start testing with all the API stuff that >>>>>> has already gone in 2.1.0. >>>>>>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I've been OOO (got a call to fix this bug), but just to clarify: We're ok with HA _not working at all_ with security enabled in 2.1.0-beta? That's the ramification of omitting HADOOP-9880. Daryn On Aug 16, 2013, at 9:20 PM, Arun C Murthy wrote: > Yep, there are quite a number of such fixes in 2.1.1 ATM, I think it will > serve us better to get 2.1.0 out and then quickly turn around to make 2.1.1. > > My current plan is to start work on 2.1.1 right after this release gets > complete… hopefully next week. > > thanks, > Arun > > On Aug 16, 2013, at 4:36 PM, Vinod Kumar Vavilapalli > wrote: > >> >> There are other such isolated and well understood bug-fixes that we pushed >> to 2.1.1 in the interesting of making progress with 2.1.0 and the >> corresponding API changes. >> >> 2.1.1 should happen soon enough after this. >> >> Thanks, >> +Vinod >> >> On Aug 16, 2013, at 4:22 PM, Roman Shaposhnik wrote: >> >>> What are the downsides of getting this fix into the 2.1? It appears >>> that the fix is pretty isolated and well understood. >>> >>> Thoughts? >>> >>> Thanks, >>> Roman. >>> >>> On Fri, Aug 16, 2013 at 3:04 PM, Kihwal Lee wrote: >>>> I've changed the target version of HADOOP-9880 to 2.1.1. Please change it >>>> back, if you feel that it needs to be in 2.1.0-beta. >>>> >>>> >>>> Kihwal >>>> >>>> >>>> ____________ >>>> From: Kihwal Lee >>>> To: Arun Murthy ; "common-dev@hadoop.apache.org" >>>> >>>> Cc: "mapreduce-...@hadoop.apache.org" ; >>>> "hdfs-...@hadoop.apache.org" ; >>>> "yarn-...@hadoop.apache.org" >>>> Sent: Friday, August 16, 2013 4:55 PM >>>> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >>>> >>>> >>>> It's your call, Arun. I.e. as long you believe rc2 meets the expectations >>>> and objectives of 2.1.0-beta. >>>> >>>> Kihwal >>>> >>>> >>>> >>>> From: Arun Murthy >>>> To: "common-dev@hadoop.apache.org" >>>> Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" >>>> ; "hdfs-...@hadoop.apache.org" >>>> ; "yarn-...@hadoop.apache.org" >>>> >>>> Sent: Friday, August 16, 2013 3:44 PM >>>> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >>>> >>>> >>>> That makes sense too. >>>> >>>> >>>> On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli >>>> wrote: >>>> >>>>> >>>>> We need to make a call on what blockers will be. From my limited >>>>> understanding, this doesn't seem like a API or a compatibility issue. Can >>>>> we not fix it in subsequent bug-fix releases? >>>>> >>>>> I do see a lot of follow up releases to 2.1.0. Getting this release out >>>>> will help downstream projects start testing with all the API stuff that >>>>> has already gone in 2.1.0. >>>>> >>>>> Thanks, >>>>> +Vinod >>>>> >>>>> On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: >>>>> >>>>>> We have found HADOOP-9880, which prevents Namenode HA from running with >>>>>> security. >>>>>> >>>>>> Kihwal >>>>>> >>>>>> >>>>>> >>>>>> From: Arun C Murthy >>>>>> To: "common-dev@hadoop.apache.org" ; >>>>>> "hdfs-...@hadoop.apache.org" ; >>>>>> "mapreduce-...@hadoop.apache.org" ; >>>>>> "yarn-...@hadoop.apache.org" >>>>>> Sent: Thursday, August 15, 2013 4:15 PM >>>>>> Subject: [VOTE] Release Apache Hadoop 2.1.0-beta >>>>>> >>>>>> >>>>>> Folks, >>>>>> >>>>>> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I >>>>>> would like to get released - this fixes the bugs we saw since the last >>>>>> go-around (rc1). >>>>>> >>>>>> The
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Yep, there are quite a number of such fixes in 2.1.1 ATM, I think it will serve us better to get 2.1.0 out and then quickly turn around to make 2.1.1. My current plan is to start work on 2.1.1 right after this release gets complete… hopefully next week. thanks, Arun On Aug 16, 2013, at 4:36 PM, Vinod Kumar Vavilapalli wrote: > > There are other such isolated and well understood bug-fixes that we pushed to > 2.1.1 in the interesting of making progress with 2.1.0 and the corresponding > API changes. > > 2.1.1 should happen soon enough after this. > > Thanks, > +Vinod > > On Aug 16, 2013, at 4:22 PM, Roman Shaposhnik wrote: > >> What are the downsides of getting this fix into the 2.1? It appears >> that the fix is pretty isolated and well understood. >> >> Thoughts? >> >> Thanks, >> Roman. >> >> On Fri, Aug 16, 2013 at 3:04 PM, Kihwal Lee wrote: >>> I've changed the target version of HADOOP-9880 to 2.1.1. Please change it >>> back, if you feel that it needs to be in 2.1.0-beta. >>> >>> >>> Kihwal >>> >>> >>> >>> From: Kihwal Lee >>> To: Arun Murthy ; "common-dev@hadoop.apache.org" >>> >>> Cc: "mapreduce-...@hadoop.apache.org" ; >>> "hdfs-...@hadoop.apache.org" ; >>> "yarn-...@hadoop.apache.org" >>> Sent: Friday, August 16, 2013 4:55 PM >>> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >>> >>> >>> It's your call, Arun. I.e. as long you believe rc2 meets the expectations >>> and objectives of 2.1.0-beta. >>> >>> Kihwal >>> >>> >>> >>> From: Arun Murthy >>> To: "common-dev@hadoop.apache.org" >>> Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" >>> ; "hdfs-...@hadoop.apache.org" >>> ; "yarn-...@hadoop.apache.org" >>> >>> Sent: Friday, August 16, 2013 3:44 PM >>> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >>> >>> >>> That makes sense too. >>> >>> >>> On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli >>> wrote: >>> >>>> >>>> We need to make a call on what blockers will be. From my limited >>>> understanding, this doesn't seem like a API or a compatibility issue. Can >>>> we not fix it in subsequent bug-fix releases? >>>> >>>> I do see a lot of follow up releases to 2.1.0. Getting this release out >>>> will help downstream projects start testing with all the API stuff that >>>> has already gone in 2.1.0. >>>> >>>> Thanks, >>>> +Vinod >>>> >>>> On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: >>>> >>>>> We have found HADOOP-9880, which prevents Namenode HA from running with >>>>> security. >>>>> >>>>> Kihwal >>>>> >>>>> >>>>> >>>>> From: Arun C Murthy >>>>> To: "common-dev@hadoop.apache.org" ; >>>>> "hdfs-...@hadoop.apache.org" ; >>>>> "mapreduce-...@hadoop.apache.org" ; >>>>> "yarn-...@hadoop.apache.org" >>>>> Sent: Thursday, August 15, 2013 4:15 PM >>>>> Subject: [VOTE] Release Apache Hadoop 2.1.0-beta >>>>> >>>>> >>>>> Folks, >>>>> >>>>> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >>>>> like to get released - this fixes the bugs we saw since the last >>>>> go-around (rc1). >>>>> >>>>> The RC is available at: >>>>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >>>>> The RC tag in svn is here: >>>>> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >>>>> >>>>> The maven artifacts are available via repository.apache.org. >>>>> >>>>> Please try the release and vote; the vote will run for the usual 7 days. >>>>> >>>>> thanks, >>>>> Arun >>>>> >>>>> -- >>>>> Arun C. Murthy >>>>> Hortonworks Inc. >>>>> http://hortonworks.com/ >>>>>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
There are other such isolated and well understood bug-fixes that we pushed to 2.1.1 in the interesting of making progress with 2.1.0 and the corresponding API changes. 2.1.1 should happen soon enough after this. Thanks, +Vinod On Aug 16, 2013, at 4:22 PM, Roman Shaposhnik wrote: > What are the downsides of getting this fix into the 2.1? It appears > that the fix is pretty isolated and well understood. > > Thoughts? > > Thanks, > Roman. > > On Fri, Aug 16, 2013 at 3:04 PM, Kihwal Lee wrote: >> I've changed the target version of HADOOP-9880 to 2.1.1. Please change it >> back, if you feel that it needs to be in 2.1.0-beta. >> >> >> Kihwal >> >> >> >> From: Kihwal Lee >> To: Arun Murthy ; "common-dev@hadoop.apache.org" >> >> Cc: "mapreduce-...@hadoop.apache.org" ; >> "hdfs-...@hadoop.apache.org" ; >> "yarn-...@hadoop.apache.org" >> Sent: Friday, August 16, 2013 4:55 PM >> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >> >> >> It's your call, Arun. I.e. as long you believe rc2 meets the expectations >> and objectives of 2.1.0-beta. >> >> Kihwal >> >> >> >> From: Arun Murthy >> To: "common-dev@hadoop.apache.org" >> Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" >> ; "hdfs-...@hadoop.apache.org" >> ; "yarn-...@hadoop.apache.org" >> >> Sent: Friday, August 16, 2013 3:44 PM >> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta >> >> >> That makes sense too. >> >> >> On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli >> wrote: >> >>> >>> We need to make a call on what blockers will be. From my limited >>> understanding, this doesn't seem like a API or a compatibility issue. Can >>> we not fix it in subsequent bug-fix releases? >>> >>> I do see a lot of follow up releases to 2.1.0. Getting this release out >>> will help downstream projects start testing with all the API stuff that has >>> already gone in 2.1.0. >>> >>> Thanks, >>> +Vinod >>> >>> On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: >>> >>>> We have found HADOOP-9880, which prevents Namenode HA from running with >>>> security. >>>> >>>> Kihwal >>>> >>>> >>>> >>>> From: Arun C Murthy >>>> To: "common-dev@hadoop.apache.org" ; >>>> "hdfs-...@hadoop.apache.org" ; >>>> "mapreduce-...@hadoop.apache.org" ; >>>> "yarn-...@hadoop.apache.org" >>>> Sent: Thursday, August 15, 2013 4:15 PM >>>> Subject: [VOTE] Release Apache Hadoop 2.1.0-beta >>>> >>>> >>>> Folks, >>>> >>>> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >>>> like to get released - this fixes the bugs we saw since the last go-around >>>> (rc1). >>>> >>>> The RC is available at: >>>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >>>> The RC tag in svn is here: >>>> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >>>> >>>> The maven artifacts are available via repository.apache.org. >>>> >>>> Please try the release and vote; the vote will run for the usual 7 days. >>>> >>>> thanks, >>>> Arun >>>> >>>> -- >>>> Arun C. Murthy >>>> Hortonworks Inc. >>>> http://hortonworks.com/ >>>> >>>> >>>> >>>> -- >>>> CONFIDENTIALITY NOTICE >>>> NOTICE: This message is intended for the use of the individual or entity to >>>> which it is addressed and may contain information that is confidential, >>>> privileged and exempt from disclosure under applicable law. If the reader >>>> of this message is not the intended recipient, you are hereby notified that >>>> any printing, copying, dissemination, distribution, disclosure or >>>> forwarding of this communication is strictly prohibited. If you have >>>> received this communication in error, please contact the sender immediately >>>> and delete it from your system. Thank You. >>> >&
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
What are the downsides of getting this fix into the 2.1? It appears that the fix is pretty isolated and well understood. Thoughts? Thanks, Roman. On Fri, Aug 16, 2013 at 3:04 PM, Kihwal Lee wrote: > I've changed the target version of HADOOP-9880 to 2.1.1. Please change it > back, if you feel that it needs to be in 2.1.0-beta. > > > Kihwal > > > > From: Kihwal Lee > To: Arun Murthy ; "common-dev@hadoop.apache.org" > > Cc: "mapreduce-...@hadoop.apache.org" ; > "hdfs-...@hadoop.apache.org" ; > "yarn-...@hadoop.apache.org" > Sent: Friday, August 16, 2013 4:55 PM > Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta > > > It's your call, Arun. I.e. as long you believe rc2 meets the expectations > and objectives of 2.1.0-beta. > > Kihwal > > > > From: Arun Murthy > To: "common-dev@hadoop.apache.org" > Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" > ; "hdfs-...@hadoop.apache.org" > ; "yarn-...@hadoop.apache.org" > > Sent: Friday, August 16, 2013 3:44 PM > Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta > > > That makes sense too. > > > On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli > wrote: > >> >> We need to make a call on what blockers will be. From my limited >> understanding, this doesn't seem like a API or a compatibility issue. Can we >> not fix it in subsequent bug-fix releases? >> >> I do see a lot of follow up releases to 2.1.0. Getting this release out will >> help downstream projects start testing with all the API stuff that has >> already gone in 2.1.0. >> >> Thanks, >> +Vinod >> >> On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: >> >>> We have found HADOOP-9880, which prevents Namenode HA from running with >>> security. >>> >>> Kihwal >>> >>> >>> >>> From: Arun C Murthy >>> To: "common-dev@hadoop.apache.org" ; >>> "hdfs-...@hadoop.apache.org" ; >>> "mapreduce-...@hadoop.apache.org" ; >>> "yarn-...@hadoop.apache.org" >>> Sent: Thursday, August 15, 2013 4:15 PM >>> Subject: [VOTE] Release Apache Hadoop 2.1.0-beta >>> >>> >>> Folks, >>> >>> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >>> like to get released - this fixes the bugs we saw since the last go-around >>> (rc1). >>> >>> The RC is available at: >>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >>> The RC tag in svn is here: >>> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >>> >>> The maven artifacts are available via repository.apache.org. >>> >>> Please try the release and vote; the vote will run for the usual 7 days. >>> >>> thanks, >>> Arun >>> >>> -- >>> Arun C. Murthy >>> Hortonworks Inc. >>> http://hortonworks.com/ >>> >>> >>> >>> -- >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity to >>> which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I've changed the target version of HADOOP-9880 to 2.1.1. Please change it back, if you feel that it needs to be in 2.1.0-beta. Kihwal From: Kihwal Lee To: Arun Murthy ; "common-dev@hadoop.apache.org" Cc: "mapreduce-...@hadoop.apache.org" ; "hdfs-...@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" Sent: Friday, August 16, 2013 4:55 PM Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta It's your call, Arun. I.e. as long you believe rc2 meets the expectations and objectives of 2.1.0-beta. Kihwal From: Arun Murthy To: "common-dev@hadoop.apache.org" Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" ; "hdfs-...@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" Sent: Friday, August 16, 2013 3:44 PM Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta That makes sense too. On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli wrote: > > We need to make a call on what blockers will be. From my limited > understanding, this doesn't seem like a API or a compatibility issue. Can we > not fix it in subsequent bug-fix releases? > > I do see a lot of follow up releases to 2.1.0. Getting this release out will > help downstream projects start testing with all the API stuff that has > already gone in 2.1.0. > > Thanks, > +Vinod > > On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: > >> We have found HADOOP-9880, which prevents Namenode HA from running with >> security. >> >> Kihwal >> >> >> >> From: Arun C Murthy >> To: "common-dev@hadoop.apache.org" ; >> "hdfs-...@hadoop.apache.org" ; >> "mapreduce-...@hadoop.apache.org" ; >> "yarn-...@hadoop.apache.org" >> Sent: Thursday, August 15, 2013 4:15 PM >> Subject: [VOTE] Release Apache Hadoop 2.1.0-beta >> >> >> Folks, >> >> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >> like to get released - this fixes the bugs we saw since the last go-around >> (rc1). >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> >> thanks, >> Arun >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
It's your call, Arun. I.e. as long you believe rc2 meets the expectations and objectives of 2.1.0-beta. Kihwal From: Arun Murthy To: "common-dev@hadoop.apache.org" Cc: Kihwal Lee ; "mapreduce-...@hadoop.apache.org" ; "hdfs-...@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" Sent: Friday, August 16, 2013 3:44 PM Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta That makes sense too. On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli wrote: > > We need to make a call on what blockers will be. From my limited > understanding, this doesn't seem like a API or a compatibility issue. Can we > not fix it in subsequent bug-fix releases? > > I do see a lot of follow up releases to 2.1.0. Getting this release out will > help downstream projects start testing with all the API stuff that has > already gone in 2.1.0. > > Thanks, > +Vinod > > On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: > >> We have found HADOOP-9880, which prevents Namenode HA from running with >> security. >> >> Kihwal >> >> >> >> From: Arun C Murthy >> To: "common-dev@hadoop.apache.org" ; >> "hdfs-...@hadoop.apache.org" ; >> "mapreduce-...@hadoop.apache.org" ; >> "yarn-...@hadoop.apache.org" >> Sent: Thursday, August 15, 2013 4:15 PM >> Subject: [VOTE] Release Apache Hadoop 2.1.0-beta >> >> >> Folks, >> >> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >> like to get released - this fixes the bugs we saw since the last go-around >> (rc1). >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> >> thanks, >> Arun >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
That makes sense too. On Aug 16, 2013, at 10:39 AM, Vinod Kumar Vavilapalli wrote: > > We need to make a call on what blockers will be. From my limited > understanding, this doesn't seem like a API or a compatibility issue. Can we > not fix it in subsequent bug-fix releases? > > I do see a lot of follow up releases to 2.1.0. Getting this release out will > help downstream projects start testing with all the API stuff that has > already gone in 2.1.0. > > Thanks, > +Vinod > > On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: > >> We have found HADOOP-9880, which prevents Namenode HA from running with >> security. >> >> Kihwal >> >> >> >> From: Arun C Murthy >> To: "common-dev@hadoop.apache.org" ; >> "hdfs-...@hadoop.apache.org" ; >> "mapreduce-...@hadoop.apache.org" ; >> "yarn-...@hadoop.apache.org" >> Sent: Thursday, August 15, 2013 4:15 PM >> Subject: [VOTE] Release Apache Hadoop 2.1.0-beta >> >> >> Folks, >> >> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >> like to get released - this fixes the bugs we saw since the last go-around >> (rc1). >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> >> thanks, >> Arun >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
We need to make a call on what blockers will be. From my limited understanding, this doesn't seem like a API or a compatibility issue. Can we not fix it in subsequent bug-fix releases? I do see a lot of follow up releases to 2.1.0. Getting this release out will help downstream projects start testing with all the API stuff that has already gone in 2.1.0. Thanks, +Vinod On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: > We have found HADOOP-9880, which prevents Namenode HA from running with > security. > > Kihwal > > > > From: Arun C Murthy > To: "common-dev@hadoop.apache.org" ; > "hdfs-...@hadoop.apache.org" ; > "mapreduce-...@hadoop.apache.org" ; > "yarn-...@hadoop.apache.org" > Sent: Thursday, August 15, 2013 4:15 PM > Subject: [VOTE] Release Apache Hadoop 2.1.0-beta > > > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Ok, I'll spin RC3. Can you please help get this in asap? Thanks, much appreciated! Arun On Aug 16, 2013, at 10:13 AM, Kihwal Lee wrote: > We have found HADOOP-9880, which prevents Namenode HA from running with > security. > > > > Kihwal > > > > From: Arun C Murthy > To: "common-dev@hadoop.apache.org" ; > "hdfs-...@hadoop.apache.org" ; > "mapreduce-...@hadoop.apache.org" ; > "yarn-...@hadoop.apache.org" > Sent: Thursday, August 15, 2013 4:15 PM > Subject: [VOTE] Release Apache Hadoop 2.1.0-beta > > > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If > you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
We have found HADOOP-9880, which prevents Namenode HA from running with security. Kihwal From: Arun C Murthy To: "common-dev@hadoop.apache.org" ; "hdfs-...@hadoop.apache.org" ; "mapreduce-...@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" Sent: Thursday, August 15, 2013 4:15 PM Subject: [VOTE] Release Apache Hadoop 2.1.0-beta Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
We have found HADOOP-9880, which prevents Namenode HA from running with security. Kihwal From: Arun C Murthy To: "common-dev@hadoop.apache.org" ; "hdfs-...@hadoop.apache.org" ; "mapreduce-...@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" Sent: Thursday, August 15, 2013 4:15 PM Subject: [VOTE] Release Apache Hadoop 2.1.0-beta Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[VOTE] Release Apache Hadoop 2.1.0-beta
Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Hi Arun, Would it be possible to include YARN-1056 in the next RC - it is a straight-forward config change. I marked it as a blocker for 2.1.0. Thanks Karthik On Thu, Aug 8, 2013 at 8:14 AM, Kihwal Lee wrote: > Another blocker, HADOOP-9850, has been committed. > > > Kihwal > > > > From: Arun C Murthy > To: Daryn Sharp > Cc: "" ; " > mapreduce-...@hadoop.apache.org" ; " > yarn-...@hadoop.apache.org" ; " > common-dev@hadoop.apache.org" > Sent: Thursday, August 1, 2013 1:30 PM > Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta > > > Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - > I'd appreciate if you could help push the fix in ASAP. > > Thanks again! > > Arun > > On Aug 1, 2013, at 9:38 AM, Daryn Sharp wrote: > > > I broke RPC QOP for integrity and privacy options. :( See blocker > HADOOP-9816. I think I understand the problem and it shouldn't be hard to > fix. > > > > The bug went unnoticed because sadly there are no unit tests for the QOP > options, even though it just involves a conf setting. > > > > Daryn > > > > > > On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote: > > > >> Ok, I think we are close to rc1 now - the last of blockers should be > committed later today… I'll try and spin RC1 tonight. > >> > >> thanks, > >> Arun > >> > >> On Jul 21, 2013, at 12:43 AM, Devaraj Das wrote: > >> > >>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. > This > >>> bug can easily be reproduced by some HBase tests. I'd like this to be > >>> considered before we make a beta release. Have spoken about this with > some > >>> hdfs folks offline and I am told that it is being worked on. > >>> > >>> Thanks > >>> Devaraj > >>> > >>> > >>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur >wrote: > >>> > >>>> As I've mentioned in my previous email, if we get YARN-701 in, we > should > >>>> also get in the fix for unmanaged AMs in an un-secure setup in > 2.1.0-beta. > >>>> Else is a regression of a functionality it is already working. > >>>> > >>>> Because of that, to avoid continuing delaying the release, I'm > suggesting > >>>> to mention in the release notes the API changes and behavior changes > that > >>>> YARN-918 and YARN-701 will bring into the next beta or GA release. > >>>> > >>>> thx > >>>> > >>>> > >>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli < > >>>> vino...@hortonworks.com> wrote: > >>>> > >>>>> > >>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: > >>>>> > >>>>>> * YARN-701 > >>>>>> > >>>>>> It should be addressed before a GA release. > >>>>>> > >>>>>> Still, as it is this breaks unmanaged AMs and to me > >>>>>> that would be a blocker for the beta. > >>>>>> > >>>>>> YARN-701 and the unmanaged AMs fix should be committed > >>>>>> in tandem. > >>>>>> > >>>>>> * YARN-918 > >>>>>> > >>>>>> It is a consequence of YARN-701 and depends on it. > >>>>> > >>>>> > >>>>> > >>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We > need > >>>>> both in 2.1.0. > >>>>> > >>>>> > >>>>> > >>>>>> * YARN-926 > >>>>>> > >>>>>> It would be nice to have it addressed before GA release. > >>>>> > >>>>> > >>>>> Either ways. I'd get it in sooner than later specifically when we are > >>>>> trying to replace the old API with the new one. > >>>>> > >>>>> Thanks, > >>>>> +Vino > >>>>> > >>>>> > >>>> > >> > >> -- > >> Arun C. Murthy > >> Hortonworks Inc. > >> http://hortonworks.com/ > >> > >> > > > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Another blocker, HADOOP-9850, has been committed. Kihwal From: Arun C Murthy To: Daryn Sharp Cc: "" ; "mapreduce-...@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" ; "common-dev@hadoop.apache.org" Sent: Thursday, August 1, 2013 1:30 PM Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP. Thanks again! Arun On Aug 1, 2013, at 9:38 AM, Daryn Sharp wrote: > I broke RPC QOP for integrity and privacy options. :( See blocker > HADOOP-9816. I think I understand the problem and it shouldn't be hard to > fix. > > The bug went unnoticed because sadly there are no unit tests for the QOP > options, even though it just involves a conf setting. > > Daryn > > > On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote: > >> Ok, I think we are close to rc1 now - the last of blockers should be >> committed later today… I'll try and spin RC1 tonight. >> >> thanks, >> Arun >> >> On Jul 21, 2013, at 12:43 AM, Devaraj Das wrote: >> >>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This >>> bug can easily be reproduced by some HBase tests. I'd like this to be >>> considered before we make a beta release. Have spoken about this with some >>> hdfs folks offline and I am told that it is being worked on. >>> >>> Thanks >>> Devaraj >>> >>> >>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur wrote: >>> >>>> As I've mentioned in my previous email, if we get YARN-701 in, we should >>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta. >>>> Else is a regression of a functionality it is already working. >>>> >>>> Because of that, to avoid continuing delaying the release, I'm suggesting >>>> to mention in the release notes the API changes and behavior changes that >>>> YARN-918 and YARN-701 will bring into the next beta or GA release. >>>> >>>> thx >>>> >>>> >>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli < >>>> vino...@hortonworks.com> wrote: >>>> >>>>> >>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: >>>>> >>>>>> * YARN-701 >>>>>> >>>>>> It should be addressed before a GA release. >>>>>> >>>>>> Still, as it is this breaks unmanaged AMs and to me >>>>>> that would be a blocker for the beta. >>>>>> >>>>>> YARN-701 and the unmanaged AMs fix should be committed >>>>>> in tandem. >>>>>> >>>>>> * YARN-918 >>>>>> >>>>>> It is a consequence of YARN-701 and depends on it. >>>>> >>>>> >>>>> >>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need >>>>> both in 2.1.0. >>>>> >>>>> >>>>> >>>>>> * YARN-926 >>>>>> >>>>>> It would be nice to have it addressed before GA release. >>>>> >>>>> >>>>> Either ways. I'd get it in sooner than later specifically when we are >>>>> trying to replace the old API with the new one. >>>>> >>>>> Thanks, >>>>> +Vino >>>>> >>>>> >>>> >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Hi Arun, Would it make sense to generate Windows binaries since this is the first 2.0 release with Windows support? I am happy to help out with that. FWIW I generated a Windows tarball from the source, brought up a pseudo-distributed cluster, ran some smoke tests and it is looking good! Thanks, Arpit On Tue, Jul 30, 2013 at 6:29 AM, Arun C Murthy wrote: > Folks, > > I've created another release candidate (rc1) for hadoop-2.1.0-beta that I > would like to get released. This RC fixes a number of issues reported on > the previous candidate. > > This release represents a *huge* amount of work done by the community > (~650 fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc1/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc1 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (non-binding) * verified md5 of the source code. * build the package from source code. * run basic hdfs commands. * run MapReduce example jobs(random text writer, wordcount, and pi) over YARN in single node. - Tsuyoshi On Tue, Jul 30, 2013 at 10:29 PM, Arun C Murthy wrote: > Folks, > > I've created another release candidate (rc1) for hadoop-2.1.0-beta that I > would like to get released. This RC fixes a number of issues reported on the > previous candidate. > > This release represents a *huge* amount of work done by the community (~650 > fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc1/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc1 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On 30 July 2013 14:29, Arun C Murthy wrote: > Folks, > > I've created another release candidate (rc1) for hadoop-2.1.0-beta that I > would like to get released. This RC fixes a number of issues reported on > the previous candidate. > > This release represents a *huge* amount of work done by the community > (~650 fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc1/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc1 > > The maven artifacts are available via repository.apache.org. > +1 set my Hoya Hbase-on-YARN app to build and run against the staging artifacts, and after catching up w/ some minor API changes, all is well -not only to the binaries work, but the POMs set up the downstream classpath with (at least) everything you need I filed one minor JIRA, "prereq check for AMRMClient.ContainerRequest relaxLocality flag wrong": https://issues.apache.org/jira/browse/YARN-1018 -Steve
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP. Thanks again! Arun On Aug 1, 2013, at 9:38 AM, Daryn Sharp wrote: > I broke RPC QOP for integrity and privacy options. :( See blocker > HADOOP-9816. I think I understand the problem and it shouldn't be hard to > fix. > > The bug went unnoticed because sadly there are no unit tests for the QOP > options, even though it just involves a conf setting. > > Daryn > > > On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote: > >> Ok, I think we are close to rc1 now - the last of blockers should be >> committed later today… I'll try and spin RC1 tonight. >> >> thanks, >> Arun >> >> On Jul 21, 2013, at 12:43 AM, Devaraj Das wrote: >> >>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This >>> bug can easily be reproduced by some HBase tests. I'd like this to be >>> considered before we make a beta release. Have spoken about this with some >>> hdfs folks offline and I am told that it is being worked on. >>> >>> Thanks >>> Devaraj >>> >>> >>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur wrote: >>> As I've mentioned in my previous email, if we get YARN-701 in, we should also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta. Else is a regression of a functionality it is already working. Because of that, to avoid continuing delaying the release, I'm suggesting to mention in the release notes the API changes and behavior changes that YARN-918 and YARN-701 will bring into the next beta or GA release. thx On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli < vino...@hortonworks.com> wrote: > > On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: > >> * YARN-701 >> >> It should be addressed before a GA release. >> >> Still, as it is this breaks unmanaged AMs and to me >> that would be a blocker for the beta. >> >> YARN-701 and the unmanaged AMs fix should be committed >> in tandem. >> >> * YARN-918 >> >> It is a consequence of YARN-701 and depends on it. > > > > YARN-918 is an API change. And YARN-701 is a behaviour change. We need > both in 2.1.0. > > > >> * YARN-926 >> >> It would be nice to have it addressed before GA release. > > > Either ways. I'd get it in sooner than later specifically when we are > trying to replace the old API with the new one. > > Thanks, > +Vino > > >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would > like to get released. > > This release represents a *huge* amount of work done by the community (639 > fixes) which > includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. +1 (non-binding). As usual I built the entire Bigtop based on this RC of Hadoop and ran the battery of smoke tests with secure and unsecure configuration. Thanks, Roman. P.S. The blocker bug from a previous RC where NN would consistently OOM seems to have disappeared. I'll update the JIRA accordingly.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix. The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting. Daryn On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote: > Ok, I think we are close to rc1 now - the last of blockers should be > committed later today… I'll try and spin RC1 tonight. > > thanks, > Arun > > On Jul 21, 2013, at 12:43 AM, Devaraj Das wrote: > >> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This >> bug can easily be reproduced by some HBase tests. I'd like this to be >> considered before we make a beta release. Have spoken about this with some >> hdfs folks offline and I am told that it is being worked on. >> >> Thanks >> Devaraj >> >> >> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur wrote: >> >>> As I've mentioned in my previous email, if we get YARN-701 in, we should >>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta. >>> Else is a regression of a functionality it is already working. >>> >>> Because of that, to avoid continuing delaying the release, I'm suggesting >>> to mention in the release notes the API changes and behavior changes that >>> YARN-918 and YARN-701 will bring into the next beta or GA release. >>> >>> thx >>> >>> >>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli < >>> vino...@hortonworks.com> wrote: >>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: > * YARN-701 > > It should be addressed before a GA release. > > Still, as it is this breaks unmanaged AMs and to me > that would be a blocker for the beta. > > YARN-701 and the unmanaged AMs fix should be committed > in tandem. > > * YARN-918 > > It is a consequence of YARN-701 and depends on it. YARN-918 is an API change. And YARN-701 is a behaviour change. We need both in 2.1.0. > * YARN-926 > > It would be nice to have it addressed before GA release. Either ways. I'd get it in sooner than later specifically when we are trying to replace the old API with the new one. Thanks, +Vino >>> > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > >
RE: [VOTE] Release Apache Hadoop 2.1.0-beta
+1. Using it locally for Tez. Bikas -Original Message- From: Arun C Murthy [mailto:a...@hortonworks.com] Sent: Tuesday, July 30, 2013 6:30 AM To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: [VOTE] Release Apache Hadoop 2.1.0-beta Folks, I've created another release candidate (rc1) for hadoop-2.1.0-beta that I would like to get released. This RC fixes a number of issues reported on the previous candidate. This release represents a *huge* amount of work done by the community (~650 fixes) which includes several major advances including: # HDFS Snapshots # Windows support # YARN API stabilization # MapReduce Binary Compatibility with hadoop-1.x # Substantial amount of integration testing with rest of projects in the ecosystem The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc1/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc1 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Mon, Jul 29, 2013 at 3:00 PM, Arun C Murthy wrote: > Ok, I think we are close to rc1 now - the last of blockers should be > committed later today… I'll try and spin RC1 tonight. Great news! I'm back from my vacation (man, those were good 3 weeks! ;-)) and I'll try to pull the RC into Bigtop and test it ASAP. Thanks, Roman.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Thanks Arun, +1 * verified MD5 & signature of source tarball. * built from source tarball * run apache-rat:check on source * installed pseudo cluster (unsecure) * test httpfs * run pi example * run unmanaged AM application Minor NITs (in case we do a new RC): * remove 2.1.1.-beta section from all CHANGES.txt files * the following hidden files shouldn't be in the source file (nor in SVN): ./.gitattributes (in SVN is OK) ./hadoop-common-project/hadoop-common/.eclipse.templates ./hadoop-common-project/hadoop-common/.eclipse.templates/.externalToolBuilders ./hadoop-hdfs-project/hadoop-hdfs/.eclipse.templates ./hadoop-hdfs-project/hadoop-hdfs/.eclipse.templates/.launches ./hadoop-mapreduce-project/.eclipse.templates ./hadoop-mapreduce-project/.eclipse.templates/.launches ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/.deps On Tue, Jul 30, 2013 at 6:29 AM, Arun C Murthy wrote: > Folks, > > I've created another release candidate (rc1) for hadoop-2.1.0-beta that I > would like to get released. This RC fixes a number of issues reported on > the previous candidate. > > This release represents a *huge* amount of work done by the community > (~650 fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc1/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc1 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > -- Alejandro
[VOTE] Release Apache Hadoop 2.1.0-beta
Folks, I've created another release candidate (rc1) for hadoop-2.1.0-beta that I would like to get released. This RC fixes a number of issues reported on the previous candidate. This release represents a *huge* amount of work done by the community (~650 fixes) which includes several major advances including: # HDFS Snapshots # Windows support # YARN API stabilization # MapReduce Binary Compatibility with hadoop-1.x # Substantial amount of integration testing with rest of projects in the ecosystem The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc1/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc1 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight. thanks, Arun On Jul 21, 2013, at 12:43 AM, Devaraj Das wrote: > I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This > bug can easily be reproduced by some HBase tests. I'd like this to be > considered before we make a beta release. Have spoken about this with some > hdfs folks offline and I am told that it is being worked on. > > Thanks > Devaraj > > > On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur wrote: > >> As I've mentioned in my previous email, if we get YARN-701 in, we should >> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta. >> Else is a regression of a functionality it is already working. >> >> Because of that, to avoid continuing delaying the release, I'm suggesting >> to mention in the release notes the API changes and behavior changes that >> YARN-918 and YARN-701 will bring into the next beta or GA release. >> >> thx >> >> >> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli < >> vino...@hortonworks.com> wrote: >> >>> >>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: >>> * YARN-701 It should be addressed before a GA release. Still, as it is this breaks unmanaged AMs and to me that would be a blocker for the beta. YARN-701 and the unmanaged AMs fix should be committed in tandem. * YARN-918 It is a consequence of YARN-701 and depends on it. >>> >>> >>> >>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need >>> both in 2.1.0. >>> >>> >>> * YARN-926 It would be nice to have it addressed before GA release. >>> >>> >>> Either ways. I'd get it in sooner than later specifically when we are >>> trying to replace the old API with the new one. >>> >>> Thanks, >>> +Vino >>> >>> >> -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This bug can easily be reproduced by some HBase tests. I'd like this to be considered before we make a beta release. Have spoken about this with some hdfs folks offline and I am told that it is being worked on. Thanks Devaraj On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur wrote: > As I've mentioned in my previous email, if we get YARN-701 in, we should > also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta. > Else is a regression of a functionality it is already working. > > Because of that, to avoid continuing delaying the release, I'm suggesting > to mention in the release notes the API changes and behavior changes that > YARN-918 and YARN-701 will bring into the next beta or GA release. > > thx > > > On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli < > vino...@hortonworks.com> wrote: > > > > > On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: > > > > > * YARN-701 > > > > > > It should be addressed before a GA release. > > > > > > Still, as it is this breaks unmanaged AMs and to me > > > that would be a blocker for the beta. > > > > > > YARN-701 and the unmanaged AMs fix should be committed > > > in tandem. > > > > > > * YARN-918 > > > > > > It is a consequence of YARN-701 and depends on it. > > > > > > > > YARN-918 is an API change. And YARN-701 is a behaviour change. We need > > both in 2.1.0. > > > > > > > > > * YARN-926 > > > > > > It would be nice to have it addressed before GA release. > > > > > > Either ways. I'd get it in sooner than later specifically when we are > > trying to replace the old API with the new one. > > > > Thanks, > > +Vino > > > > >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: > * YARN-701 > > It should be addressed before a GA release. > > Still, as it is this breaks unmanaged AMs and to me > that would be a blocker for the beta. > > YARN-701 and the unmanaged AMs fix should be committed > in tandem. > > * YARN-918 > > It is a consequence of YARN-701 and depends on it. YARN-918 is an API change. And YARN-701 is a behaviour change. We need both in 2.1.0. > * YARN-926 > > It would be nice to have it addressed before GA release. Either ways. I'd get it in sooner than later specifically when we are trying to replace the old API with the new one. Thanks, +Vino
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Vinod, Thanks for reviving this thread. The current blockers are: https://issues.apache.org/jira/issues/?jql=project%20in%20(hadoop%2C%20mapreduce%2C%20hdfs%2C%20yarn)%20and%20status%20in%20(open%2C%20'patch%20available')%20and%20priority%20%3D%20blocker%20and%20%22Target%20Version%2Fs%22%20%3D%20%222.1.0-beta%22 By looking at them I don't see they are necessary blockers for a beta release. * HADOOP-9688 & HADOOP-9698 They definitely have to be addressed before a GA release. * YARN-701 It should be addressed before a GA release. Still, as it is this breaks unmanaged AMs and to me that would be a blocker for the beta. YARN-701 and the unmanaged AMs fix should be committed in tandem. * YARN-918 It is a consequence of YARN-701 and depends on it. * YARN-926 It would be nice to have it addressed before GA release. We could do a beta with what we have at the moment in branch-2 and have a special release note indicating API changes coming in the next beta/GA release as part of YARN-918 & YARN-926. IMO, we should move forward with the beta release with the current state. Else we'll continue delaying it and adding more things that break/change things. Thanks. Alejandro On Wed, Jul 17, 2013 at 12:24 PM, Vinod Kumar Vavilapalli < vino...@hortonworks.com> wrote: > > Looks like this RC has gone stale and lots of bug fixes went into 2.1 and > 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see > in CHANGES.txt files there seems to be a confusion about which branch to > get in what. > > I'm blowing off the current 2.1.0 release branch so that we can create a > fresh release branch and call voting on that. I'll fix CHANGES.txt entries > as well as JIRA fix version for bugs committed recently if there are > inconsistencies. > > Let me know if something is amiss while I do this. > > Thanks, > +Vinod > > On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote: > > > > > We should get these in, looking at them now. > > > > Thanks, > > +Vinod > > > > On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote: > > > >> Hi Arun, > >> > >> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may > potentially change the apis. They can implemented in a backward compat > fashion if committed after 2.1.0. However, this will require adding of > differently-named apis ( different urls in case of the webservices ) and > make the current version of the api deprecated and/or obsolete. YARN-818 > which is currently patch available also changes behavior. > >> > >> Assuming that as soon as 2.1.0 is released, we are to follow a very > strict backward-compat retaining approach to all user-facing layers ( > api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense > to try and pull them in and roll out a new RC after they are ready? Perhaps > Vinod can chime in if he is aware of any other such jiras under YARN-386 > which should be considered compat-related blockers for a 2.1.0 RC. > >> > >> thanks > >> -- Hitesh > >> > >> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote: > >> > >>> Folks, > >>> > >>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I > would like to get released. > >>> > >>> This release represents a *huge* amount of work done by the community > (639 fixes) which includes several major advances including: > >>> # HDFS Snapshots > >>> # Windows support > >>> # YARN API stabilization > >>> # MapReduce Binary Compatibility with hadoop-1.x > >>> # Substantial amount of integration testing with rest of projects in > the ecosystem > >>> > >>> The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ > >>> The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > >>> > >>> The maven artifacts are available via repository.apache.org. > >>> > >>> Please try the release and vote; the vote will run for the usual 7 > days. > >>> > >>> thanks, > >>> Arun > >>> > >>> -- > >>> Arun C. Murthy > >>> Hortonworks Inc. > >>> http://hortonworks.com/ > >>> > >>> > >> > > > > -- Alejandro
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Looks like this RC has gone stale and lots of bug fixes went into 2.1 and 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see in CHANGES.txt files there seems to be a confusion about which branch to get in what. I'm blowing off the current 2.1.0 release branch so that we can create a fresh release branch and call voting on that. I'll fix CHANGES.txt entries as well as JIRA fix version for bugs committed recently if there are inconsistencies. Let me know if something is amiss while I do this. Thanks, +Vinod On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote: > > We should get these in, looking at them now. > > Thanks, > +Vinod > > On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote: > >> Hi Arun, >> >> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may >> potentially change the apis. They can implemented in a backward compat >> fashion if committed after 2.1.0. However, this will require adding of >> differently-named apis ( different urls in case of the webservices ) and >> make the current version of the api deprecated and/or obsolete. YARN-818 >> which is currently patch available also changes behavior. >> >> Assuming that as soon as 2.1.0 is released, we are to follow a very strict >> backward-compat retaining approach to all user-facing layers ( >> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense >> to try and pull them in and roll out a new RC after they are ready? Perhaps >> Vinod can chime in if he is aware of any other such jiras under YARN-386 >> which should be considered compat-related blockers for a 2.1.0 RC. >> >> thanks >> -- Hitesh >> >> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote: >> >>> Folks, >>> >>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would >>> like to get released. >>> >>> This release represents a *huge* amount of work done by the community (639 >>> fixes) which includes several major advances including: >>> # HDFS Snapshots >>> # Windows support >>> # YARN API stabilization >>> # MapReduce Binary Compatibility with hadoop-1.x >>> # Substantial amount of integration testing with rest of projects in the >>> ecosystem >>> >>> The RC is available at: >>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ >>> The RC tag in svn is here: >>> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 >>> >>> The maven artifacts are available via repository.apache.org. >>> >>> Please try the release and vote; the vote will run for the usual 7 days. >>> >>> thanks, >>> Arun >>> >>> -- >>> Arun C. Murthy >>> Hortonworks Inc. >>> http://hortonworks.com/ >>> >>> >> >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I committed MAPREDUCE-5358 to branch-2 but did not commit it to branch-2.1-beta since it wasn't a blocker and Arun was in the middle of cutting the release. Arun, if you feel it's appropriate to put this in branch-2.1-beta feel free to pull it in or let me know. Thanks! Jason On 07/03/2013 02:24 AM, Tsuyoshi OZAWA wrote: Hi Arun, Some bug fixes about MapReduce should be included in next release. MAPREDUCE-5221 should be merged because it's a lost feature of MRv2 compared to MRv1. MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be merged immediately. # I sent the previous email only to hdfs-dev, so I resending it. # I apologize for my mistake. On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released. This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including: # HDFS Snapshots # Windows support # YARN API stabilization # MapReduce Binary Compatibility with hadoop-1.x # Substantial amount of integration testing with rest of projects in the ecosystem The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
We should get these in, looking at them now. Thanks, +Vinod On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote: > Hi Arun, > > From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may > potentially change the apis. They can implemented in a backward compat > fashion if committed after 2.1.0. However, this will require adding of > differently-named apis ( different urls in case of the webservices ) and make > the current version of the api deprecated and/or obsolete. YARN-818 which is > currently patch available also changes behavior. > > Assuming that as soon as 2.1.0 is released, we are to follow a very strict > backward-compat retaining approach to all user-facing layers ( > api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense > to try and pull them in and roll out a new RC after they are ready? Perhaps > Vinod can chime in if he is aware of any other such jiras under YARN-386 > which should be considered compat-related blockers for a 2.1.0 RC. > > thanks > -- Hitesh > > On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote: > >> Folks, >> >> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would >> like to get released. >> >> This release represents a *huge* amount of work done by the community (639 >> fixes) which includes several major advances including: >> # HDFS Snapshots >> # Windows support >> # YARN API stabilization >> # MapReduce Binary Compatibility with hadoop-1.x >> # Substantial amount of integration testing with rest of projects in the >> ecosystem >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> >> thanks, >> Arun >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Hi Arun, Some bug fixes about MapReduce should be included in next release. MAPREDUCE-5221 should be merged because it's a lost feature of MRv2 compared to MRv1. MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be merged immediately. # I sent the previous email only to hdfs-dev, so I resending it. # I apologize for my mistake. On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would > like to get released. > > This release represents a *huge* amount of work done by the community (639 > fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > -- - Tsuyoshi
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
-1. Some of the cli and distcp system tests which use hftp:// and webhdfs:// are failing on secure cluster (HDFS-4841 and HDFS-4952/HDFS-4896). This is a regression and we need to make sure they work before we call a release. On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would > like to get released. > > This release represents a *huge* amount of work done by the community (639 > fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Hi Arun! On Fri, Jun 28, 2013 at 2:05 PM, Arun C Murthy wrote: > Thanks Hitesh & Roman, I'll roll RC1 once these are fixed. Here's the latest update. I've incorporated Collin's patch for HADOOP-9676 and re-ran the tests. A few subtests of TestCLI still fail, but quite a few pass and on top of that the failing ones no longer make the NN go OOM. Now, we still need to get to the bottom of why the subset of tests keep failing, but at this point I'd feel comfortable committing a change for HADOOP-9676 to branch-2.1 and going from there. Let me know if I can help some more. Thanks, Roman.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Thanks Hitesh & Roman, I'll roll RC1 once these are fixed. Arun On Jun 28, 2013, at 12:25 PM, Roman Shaposhnik wrote: > I think it would very useful if somebody more familiar with HDFS > (Suresh?) could take a look at: > https://issues.apache.org/jira/browse/HDFS-4940 > > Suresh was very helpful in unblocking the client side of things, but > even after I followed his recommendations on the heap size, etc. > The NN still OOMs it just takes longer. > > A quick look at the provided heap dump could help us understand > whether this is something serious or not. > > This is the last bit that blocks the Bigtop side of things as far as > 2.1 is concerned. > > Thanks, > Roman. > > On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy wrote: >> Folks, >> >> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would >> like to get released. >> >> This release represents a *huge* amount of work done by the community (639 >> fixes) which includes several major advances including: >> # HDFS Snapshots >> # Windows support >> # YARN API stabilization >> # MapReduce Binary Compatibility with hadoop-1.x >> # Substantial amount of integration testing with rest of projects in the >> ecosystem >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> >> thanks, >> Arun >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I think it would very useful if somebody more familiar with HDFS (Suresh?) could take a look at: https://issues.apache.org/jira/browse/HDFS-4940 Suresh was very helpful in unblocking the client side of things, but even after I followed his recommendations on the heap size, etc. The NN still OOMs it just takes longer. A quick look at the provided heap dump could help us understand whether this is something serious or not. This is the last bit that blocks the Bigtop side of things as far as 2.1 is concerned. Thanks, Roman. On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would > like to get released. > > This release represents a *huge* amount of work done by the community (639 > fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
For YARN-791, if we can come to consensus on the correct approach, I can try to have a patch ASAP. -Sandy On Fri, Jun 28, 2013 at 12:03 PM, Hitesh Shah wrote: > Hi Arun, > > From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may > potentially change the apis. They can implemented in a backward compat > fashion if committed after 2.1.0. However, this will require adding of > differently-named apis ( different urls in case of the webservices ) and > make the current version of the api deprecated and/or obsolete. YARN-818 > which is currently patch available also changes behavior. > > Assuming that as soon as 2.1.0 is released, we are to follow a very strict > backward-compat retaining approach to all user-facing layers ( > api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense > to try and pull them in and roll out a new RC after they are ready? Perhaps > Vinod can chime in if he is aware of any other such jiras under YARN-386 > which should be considered compat-related blockers for a 2.1.0 RC. > > thanks > -- Hitesh > > On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote: > > > Folks, > > > > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I > would like to get released. > > > > This release represents a *huge* amount of work done by the community > (639 fixes) which includes several major advances including: > > # HDFS Snapshots > > # Windows support > > # YARN API stabilization > > # MapReduce Binary Compatibility with hadoop-1.x > > # Substantial amount of integration testing with rest of projects in the > ecosystem > > > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ > > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > > > > The maven artifacts are available via repository.apache.org. > > > > Please try the release and vote; the vote will run for the usual 7 days. > > > > thanks, > > Arun > > > > -- > > Arun C. Murthy > > Hortonworks Inc. > > http://hortonworks.com/ > > > > > >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Hi Arun, From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior. Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC. thanks -- Hitesh On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would > like to get released. > > This release represents a *huge* amount of work done by the community (639 > fixes) which includes several major advances including: > # HDFS Snapshots > # Windows support > # YARN API stabilization > # MapReduce Binary Compatibility with hadoop-1.x > # Substantial amount of integration testing with rest of projects in the > ecosystem > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > >
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On 26 June 2013 09:17, Arun C Murthy wrote: > Folks, > > > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 > > The maven artifacts are available via repository.apache.org. > > did all the poms go up? [INFO] [INFO] Building Hoya: HBase on YARN 0.0.1-SNAPSHOT [INFO] Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.pom [WARNING] The POM for org.apache.hadoop:hadoop-common:jar:2.1.0-beta is missing, no dependency information available Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.pom [WARNING] The POM for org.apache.hadoop:hadoop-minicluster:jar:2.1.0-beta is missing, no dependency information available Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.pom [WARNING] The POM for org.apache.hadoop:hadoop-hdfs:jar:2.1.0-beta is missing, no dependency information available [WARNING] The POM for org.apache.hadoop:hadoop-common:jar:tests:2.1.0-beta is missing, no dependency information available Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.pom [WARNING] The POM for org.apache.hadoop:hadoop-yarn-server-common:jar:2.1.0-beta is missing, no dependency information available Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.jar Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.jar Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.jar Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta-tests.jar Downloading: http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.jar [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 2.137s [INFO] Finished at: Wed Jun 26 20:45:22 BST 2013 [INFO] Final Memory: 6M/265M [INFO] [ERROR] Failed to execute goal on project hoya: Could not resolve dependencies for project org.apache.hadoop.hoya:hoya:jar:0.0.1-SNAPSHOT: The following artifacts could not be resolved: org.apache.hadoop:hadoop-common:jar:2.1.0-beta, org.apache.hadoop:hadoop-minicluster:jar:2.1.0-beta, org.apache.hadoop:hadoop-hdfs:jar:2.1.0-beta, org.apache.hadoop:hadoop-common:jar:tests:2.1.0-beta, org.apache.hadoop:hadoop-yarn-server-common:jar:2.1.0-beta: Could not find artifact org.apache.hadoop:hadoop-common:jar:2.1.0-beta in central ( http://repo.maven.apache.org/maven2) -> [Help 1]
[VOTE] Release Apache Hadoop 2.1.0-beta
Folks, I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released. This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including: # HDFS Snapshots # Windows support # YARN API stabilization # MapReduce Binary Compatibility with hadoop-1.x # Substantial amount of integration testing with rest of projects in the ecosystem The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
