Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (non-binding) Built from source, ran jobs on a pseudo-distributed cluster with the Fair Scheduler. -Sandy On Tue, Aug 20, 2013 at 7:27 PM, Arun C Murthy a...@hortonworks.com wrote: Thanks for the heads up Aaron, I've changed fix-version of HDFS-4763 to 2.1.1-beta for now. Committers - please be careful setting fix-versions, this is a good anti-pattern to avoid… though, I'm willing to bet a lot of dough that this isn't the first Hadoop release with this issue… *smile* Arun On Aug 20, 2013, at 6:09 PM, Aaron T. Myers a...@cloudera.com wrote: I was evaluating the release bits when I noticed that the change done in HDFS-4763 to add support for starting the HDFS NFSv3 gateway, which is marked with a fix version of 2.1.0-beta and included in the release notes of RC2, is not in fact included in the RC2 release bits. It looks to me like the change is included in branch-2.1-beta, but not branch-2.1.0-beta. Particularly since the release notes in RC2 are incorrect in claiming that this change is in this release, it seems like a pretty serious issue. Ordinarily I'd say that this issue should result in a new RC, and I would vote -1 on RC2. But, given the previous discussion that folks are interested in releasing 2.1.0-beta with several fairly substantial bugs that we already know about, I'll withhold my vote. If RC2 ends up getting released as-is, we should be sure to change the fix version field on that JIRA to be correct. -- Aaron T. Myers Software Engineer, Cloudera On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)
+1 I downloaded and ran some examples, It works fine. Thanks Devaraj k -Original Message- From: Konstantin Boudnik [mailto:c...@apache.org] Sent: 16 August 2013 11:00 To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1) All, I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would like to release. This is a stabilization release that includes fixed for a couple a of issues as outlined on the security list. The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1 The maven artifacts are available via repository.apache.org. The only difference between rc0 and rc1 is ASL added to releasenotes.html and updated release dates in CHANGES.txt files. Please try the release bits and vote; the vote will run for the usual 7 days. Thanks for your voting Cos
[jira] [Created] (HADOOP-9893) Ticket cache support for MiniKdc
Kai Zheng created HADOOP-9893: - Summary: Ticket cache support for MiniKdc Key: HADOOP-9893 URL: https://issues.apache.org/jira/browse/HADOOP-9893 Project: Hadoop Common Issue Type: Improvement Reporter: Kai Zheng Assignee: Kai Zheng As discussed in HADOOP-9881, it would be good to support ticket cache generation utilizing MiniKdc, which can be used to test some Kerberos cases for UGI regarding user login via kinit or ticket cache. Currently it's not supported and this issue is to implement it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9877) Fix listing of snapshot directories in globStatus
[
https://issues.apache.org/jira/browse/HADOOP-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang resolved HADOOP-9877.
-
Resolution: Fixed
Target Version/s: 2.3.0 (was: 2.1.0-beta)
+1 for branch-2 patch, committed. Thanks Binglin for the quick turnaround!
Fix listing of snapshot directories in globStatus
-
Key: HADOOP-9877
URL: https://issues.apache.org/jira/browse/HADOOP-9877
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.1.0-beta
Reporter: Binglin Chang
Assignee: Binglin Chang
Fix For: 3.0.0, 2.3.0
Attachments: HADOOP-9877-branch2.patch, HADOOP-9877.v1.patch,
HADOOP-9877.v2.patch, HADOOP-9877.v3.patch, HADOOP-9877.v4.patch,
HADOOP-9877.v5.patch
{code}
decster:~/hadoop bin/hadoop fs -ls /foo/.snapshot
13/08/16 01:17:22 INFO hdfs.DFSClient: + listPath(/)
13/08/16 01:17:22 INFO hdfs.DFSClient: + listPath(/foo)
ls: `/foo/.snapshot': No such file or directory
{code}
HADOOP-9817 refactor some globStatus code, but forgot to handle special case
that .snapshot dir is not show up in listStatus but exists, so we need to
explicitly check path existence using getFileStatus, rather than depending on
listStatus results.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9894) Race condition in Shell leads to logged error stream handling exceptions
Jason Lowe created HADOOP-9894: -- Summary: Race condition in Shell leads to logged error stream handling exceptions Key: HADOOP-9894 URL: https://issues.apache.org/jira/browse/HADOOP-9894 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 2.1.0-beta Reporter: Jason Lowe Shell.runCommand starts an error stream handling thread and normally joins with it before closing the error stream. However if parseExecResult throws an exception (e.g.: like Stat.parseExecResult does for FileNotFoundException) then the error thread is not joined and the error stream can be closed before the error stream handling thread is finished. This causes the error stream handling thread to log an exception backtrace for a normal situation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (binding). Verified signature and checksums. Built from source and upgraded a 2.0.4 cluster. Ran a couple of simple MR and Tez jobs. - Sid On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HADOOP-9895) NativeCodeLoader incorrectly returns
Colin Patrick McCabe created HADOOP-9895: Summary: NativeCodeLoader incorrectly returns Key: HADOOP-9895 URL: https://issues.apache.org/jira/browse/HADOOP-9895 Project: Hadoop Common Issue Type: Bug Reporter: Colin Patrick McCabe -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
St.Ack wrote: + Once I figured where the logs were, found that JAVA_HOME was not being exported (don't need this in hadoop-2.0.5 for instance). Adding an exported JAVA_HOME to my running shell which don't seem right but it took care of it (I gave up pretty quick on messing w/ yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't getting anywhere) I thought that we were always supposed to have JAVA_HOME set when running any of these commands. At least, I do. How else can the system disambiguate between different Java installs? I need 2 installs to test with JDK7. + This did not seem to work for me: namehadoop.security.group.mapping/name valueorg.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback/va lue. We've seen this before. I think your problem is that you have java.library.path set correctly (what System.loadLibrary checks), but your system library path does not include a necessary dependency of libhadoop.so-- most likely, libjvm.so. Probably, we should fix NativeCodeLoader to actually make a function call in libhadoop.so before it declares everything OK. Colin On Tue, Aug 20, 2013 at 5:35 PM, Stack st...@duboce.net wrote: On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. It basically works (in insecure mode), +1. + Checked signature. + Ran on small cluster w/ small load made using mapreduce interfaces. + Got the HBase full unit test suite to pass on top of it. I had the following issues getting it to all work. I don't know if they are known issues so will just list them here first. + I could not find documentation on how to go from tarball to running cluster (the bundled 'cluster' and 'standalone' doc are not about how to get this tarball off the ground). + I had a bit of a struggle putting this release in place under hbase unit tests. The container would just exit w/ 127 errcode. No logs in expected place. Tripped over where minimrcluster was actually writing. Tried to corral it so it played nicely w/o our general test setup but found that the new mini clusters have 'target' hardcoded as output dirs. + Once I figured where the logs were, found that JAVA_HOME was not being exported (don't need this in hadoop-2.0.5 for instance). Adding an exported JAVA_HOME to my running shell which don't seem right but it took care of it (I gave up pretty quick on messing w/ yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't getting anywhere) + This did not seem to work for me: namehadoop.security.group.mapping/name valueorg.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback/value. It just did this: Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.clinit(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.init(JniBasedUnixGroupsMappingWithFallback.java:38) ..so I replaced it w/ org.apache.hadoop.security.ShellBasedUnixGroupsMapping on the hbase-side to get my cluster up and running. + Untarring the bin dir, it undoes as hadoop-X.Y.Z-beta. Undoing the src dir it undoes as hadoop-X.Y.Z-beta-src. I'd have thought they would undo into the one directory overlaying each other. St.Ack
Re: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)
+1 (binding) I downloaded the bits, set up a 4-node cluster, and ran some example jobs. Looks good to me. -- Aaron T. Myers Software Engineer, Cloudera On Thu, Aug 15, 2013 at 10:29 PM, Konstantin Boudnik c...@apache.org wrote: All, I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would like to release. This is a stabilization release that includes fixed for a couple a of issues as outlined on the security list. The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1 The maven artifacts are available via repository.apache.org. The only difference between rc0 and rc1 is ASL added to releasenotes.html and updated release dates in CHANGES.txt files. Please try the release bits and vote; the vote will run for the usual 7 days. Thanks for your voting Cos
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Wed, Aug 21, 2013 at 1:25 PM, Colin McCabe cmcc...@alumni.cmu.eduwrote: St.Ack wrote: + Once I figured where the logs were, found that JAVA_HOME was not being exported (don't need this in hadoop-2.0.5 for instance). Adding an exported JAVA_HOME to my running shell which don't seem right but it took care of it (I gave up pretty quick on messing w/ yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't getting anywhere) I thought that we were always supposed to have JAVA_HOME set when running any of these commands. At least, I do. How else can the system disambiguate between different Java installs? I need 2 installs to test with JDK7. That is fair enough but I did not need to define this explicitly previously (for hadoop-2.0.5-alpha for instance) or the JAVA_HOME that was figured in start scripts was propagated and now is not (I have not dug in). + This did not seem to work for me: namehadoop.security.group.mapping/name valueorg.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback/va lue. We've seen this before. I think your problem is that you have java.library.path set correctly (what System.loadLibrary checks), but your system library path does not include a necessary dependency of libhadoop.so-- most likely, libjvm.so. Probably, we should fix NativeCodeLoader to actually make a function call in libhadoop.so before it declares everything OK. My expectation was that if native group lookup fails, as it does here, then the 'Fallback' would kick in and we'd do the Shell query. This mechanism does not seem to be working. St.Ack
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (binding) - verified signatures and checksums - built from source - ran some simple jobs on a single-node cluster On 08/15/2013 04:15 PM, Arun C Murthy wrote: Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)
+1 (binding) - Verified signatures and checksums - Built source - Deployed single-node cluster and ran some test jobs Jason On 08/16/2013 12:29 AM, Konstantin Boudnik wrote: All, I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would like to release. This is a stabilization release that includes fixed for a couple a of issues as outlined on the security list. The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1 The maven artifacts are available via repository.apache.org. The only difference between rc0 and rc1 is ASL added to releasenotes.html and updated release dates in CHANGES.txt files. Please try the release bits and vote; the vote will run for the usual 7 days. Thanks for your voting Cos
[jira] [Created] (HADOOP-9896) TestIPC fail with VM crash or System.exit
shanyu zhao created HADOOP-9896: --- Summary: TestIPC fail with VM crash or System.exit Key: HADOOP-9896 URL: https://issues.apache.org/jira/browse/HADOOP-9896 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.0.5-alpha Reporter: shanyu zhao Attachments: org.apache.hadoop.ipc.TestIPC-output.txt I'm running hadoop unit tests on a Ubuntu 12.04 virtual machine, every time I try to run all unit tests with command mvn test, the TestIPC unit test will fail, the console will show The forked VM terminated without saying properly goodbye. VM crash or System.exit called? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
+1 (binding) Verified checksums, built from source and ran basic MR jobs on a single-node cluster. -- Hitesh On Aug 15, 2013, at 2:15 PM, Arun C Murthy wrote: Folks, I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would like to get released - this fixes the bugs we saw since the last go-around (rc1). The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
