[jira] [Created] (HADOOP-9888) KerberosName static initialization gets default realm, which is unneeded in non-secure deployment.
Chris Nauroth created HADOOP-9888:
-
Summary: KerberosName static initialization gets default realm,
which is unneeded in non-secure deployment.
Key: HADOOP-9888
URL: https://issues.apache.org/jira/browse/HADOOP-9888
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 3.0.0, 2.1.1-beta
Reporter: Chris Nauroth
{{KerberosName}} has a static initialization block that looks up the default
realm. Running with Oracle JDK7, this code path triggers a DNS query. In some
environments, we've seen this DNS query block and time out after 30 seconds.
This is part of static initialization, and the class is referenced from
{{UserGroupInformation#initialize}}, so every daemon and every shell command
experiences this delay. This occurs even for non-secure deployments, which
don't need the default realm.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9693) Shell should add a probe for OSX
[
https://issues.apache.org/jira/browse/HADOOP-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-9693.
Resolution: Fixed
Fix Version/s: 2.3.0
3.0.0
fixed in branch-2 and trunk, not 2.1-beta
> Shell should add a probe for OSX
>
>
> Key: HADOOP-9693
> URL: https://issues.apache.org/jira/browse/HADOOP-9693
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Steve Loughran
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
>
> the {{Shell}} class looks for Windows and Linux and sets static fields to
> show this. For tests, it's sometimes useful to know if a host is OS/X, as its
> filesystem is different. Although this is easy to test in code
> [https://developer.apple.com/library/mac/#technotes/tn2002/tn2110.html],
> having this in one place makes it easier to maintain if future java versions
> ever behave differently
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Hadoop is in Fedora
The hadoop package has passed review and has been built for Fedora 20. Rob
[jira] [Created] (HADOOP-9889) Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC
Wei Yan created HADOOP-9889: --- Summary: Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC Key: HADOOP-9889 URL: https://issues.apache.org/jira/browse/HADOOP-9889 Project: Hadoop Common Issue Type: Bug Reporter: Wei Yan Assignee: Wei Yan Krb5 Config uses a singleton and once initialized it does not refresh automatically. Without refresh, there are failures if you are using MiniKDCs with different configurations (such as different realms) within the same test run or if the Krb5 Config singleton is called before the MiniKDC is started for the first time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Hadoop is in Fedora
On 08/20/2013 04:35 PM, Robert Rati wrote: The hadoop package has passed review and has been built for Fedora 20. Rob Well done!
[jira] [Created] (HADOOP-9890) single cluster setup docs donbroken
Steve Loughran created HADOOP-9890: -- Summary: single cluster setup docs donbroken Key: HADOOP-9890 URL: https://issues.apache.org/jira/browse/HADOOP-9890 Project: Hadoop Common Issue Type: Bug Reporter: Steve Loughran Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9891) CLIMiniCluster instructions fail with MiniYarnCluster ClassNotFound
Steve Loughran created HADOOP-9891:
--
Summary: CLIMiniCluster instructions fail with MiniYarnCluster
ClassNotFound
Key: HADOOP-9891
URL: https://issues.apache.org/jira/browse/HADOOP-9891
Project: Hadoop Common
Issue Type: Bug
Components: documentation
Affects Versions: 2.1.1-beta
Reporter: Steve Loughran
Priority: Minor
The instruction on how to start up a mini CLI cluster in {{CLIMiniCluster.md}}
don't work -it looks like {{MiniYarnCluster}} isn't on the classpath
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)
+1 Did the same as with rc0. Works for me. Thanks, --Konst On Thu, Aug 15, 2013 at 10:29 PM, Konstantin Boudnik wrote: > All, > > I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I > would > like to release. > > This is a stabilization release that includes fixed for a couple a of > issues > as outlined on the security list. > > The RC is available at: > http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1 > > The maven artifacts are available via repository.apache.org. > > The only difference between rc0 and rc1 is ASL added to releasenotes.html > and > updated release dates in CHANGES.txt files. > > Please try the release bits and vote; the vote will run for the usual 7 > days. > > Thanks for your voting > Cos > >
[jira] [Reopened] (HADOOP-9877) Fix listing of snapshot directories in globStatus
[
https://issues.apache.org/jira/browse/HADOOP-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang reopened HADOOP-9877:
-
Hey Binglin, turns out this doesn't apply cleanly to branch-2 (had to revert it
out).
Do you mind submitting a branch-2 version as well?
> Fix listing of snapshot directories in globStatus
> -
>
> Key: HADOOP-9877
> URL: https://issues.apache.org/jira/browse/HADOOP-9877
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.1.0-beta
>Reporter: Binglin Chang
>Assignee: Binglin Chang
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HADOOP-9877.v1.patch, HADOOP-9877.v2.patch,
> HADOOP-9877.v3.patch, HADOOP-9877.v4.patch, HADOOP-9877.v5.patch
>
>
> {code}
> decster:~/hadoop> bin/hadoop fs -ls "/foo/.snapshot"
> 13/08/16 01:17:22 INFO hdfs.DFSClient: + listPath(/)
> 13/08/16 01:17:22 INFO hdfs.DFSClient: + listPath(/foo)
> ls: `/foo/.snapshot': No such file or directory
> {code}
> HADOOP-9817 refactor some globStatus code, but forgot to handle special case
> that .snapshot dir is not show up in listStatus but exists, so we need to
> explicitly check path existence using getFileStatus, rather than depending on
> listStatus results.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > It basically works (in insecure mode), +1. + Checked signature. + Ran on small cluster w/ small load made using mapreduce interfaces. + Got the HBase full unit test suite to pass on top of it. I had the following issues getting it to all work. I don't know if they are known issues so will just list them here first. + I could not find documentation on how to go from tarball to running cluster (the bundled 'cluster' and 'standalone' doc are not about how to get this tarball off the ground). + I had a bit of a struggle putting this release in place under hbase unit tests. The container would just exit w/ 127 errcode. No logs in expected place. Tripped over where minimrcluster was actually writing. Tried to corral it so it played nicely w/o our general test setup but found that the new mini clusters have 'target' hardcoded as output dirs. + Once I figured where the logs were, found that JAVA_HOME was not being exported (don't need this in hadoop-2.0.5 for instance). Adding an exported JAVA_HOME to my running shell which don't seem right but it took care of it (I gave up pretty quick on messing w/ yarn.nodemanager.env-whitelist and yarn.nodemanager.admin-env -- I wasn't getting anywhere) + This did not seem to work for me: hadoop.security.group.mapping org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback. It just did this: Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.(JniBasedUnixGroupsMappingWithFallback.java:38) ..so I replaced it w/ org.apache.hadoop.security.ShellBasedUnixGroupsMapping on the hbase-side to get my cluster up and running. + Untarring the bin dir, it undoes as hadoop-X.Y.Z-beta. Undoing the src dir it undoes as hadoop-X.Y.Z-beta-src. I'd have thought they would undo into the one directory overlaying each other. St.Ack
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
I was evaluating the release bits when I noticed that the change done in HDFS-4763 to add support for starting the HDFS NFSv3 gateway, which is marked with a "fix version" of 2.1.0-beta and included in the release notes of RC2, is not in fact included in the RC2 release bits. It looks to me like the change is included in branch-2.1-beta, but not branch-2.1.0-beta. Particularly since the release notes in RC2 are incorrect in claiming that this change is in this release, it seems like a pretty serious issue. Ordinarily I'd say that this issue should result in a new RC, and I would vote -1 on RC2. But, given the previous discussion that folks are interested in releasing 2.1.0-beta with several fairly substantial bugs that we already know about, I'll withhold my vote. If RC2 ends up getting released as-is, we should be sure to change the fix version field on that JIRA to be correct. -- Aaron T. Myers Software Engineer, Cloudera On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since the last go-around > (rc1). > > The RC is available at: > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > The maven artifacts are available via repository.apache.org. > > Please try the release and vote; the vote will run for the usual 7 days. > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
Re: hadoop-2.1.1-beta & hadoop-2.2.0 (GA)
FYI - here is the end-point I'm using to track blockers on 2.1.1-beta: http://s.apache.org/hadoop-2.1.1-beta-blockers Essentially, these are *Blocker* bugs with *Target Version* set to 2.1.1-beta. thanks, Arun On Aug 16, 2013, at 7:39 PM, Arun C Murthy wrote: > Gang, > > I spent time looking through changes slated for hadoop-2.1.1-beta and things > look fairly contained (~10 or so changes for each of Common, HDFS, YARN & > MapReduce). > > Can I, henceforth, request committers to exercise large dollops of caution > when committing to branch-2.1-beta? This way I hope we can quickly turn > around to make a hadoop-2.1.1-beta release in the next couple of weeks. This > can be followed by a bit more testing so that we are in a position to release > hadoop-2.2.0 (GA/stable). > > In other words, I'm hoping hadoop-2.1.1-beta can be, um, the 'golden master' > for the final hadoop-2 GA release. I don't mean to jinx it but saying it out > loud, but I feel we could look at pushing out hadoop-2 GA by mid-September… > there I said it! *smile* > > Thoughts? > > thanks, > Arun > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Thanks for the heads up Aaron, I've changed fix-version of HDFS-4763 to 2.1.1-beta for now. Committers - please be careful setting fix-versions, this is a good anti-pattern to avoid… though, I'm willing to bet a lot of dough that this isn't the first Hadoop release with this issue… *smile* Arun On Aug 20, 2013, at 6:09 PM, Aaron T. Myers wrote: > I was evaluating the release bits when I noticed that the change done in > HDFS-4763 to add support for starting the HDFS NFSv3 gateway, which is > marked with a "fix version" of 2.1.0-beta and included in the release notes > of RC2, is not in fact included in the RC2 release bits. It looks to me > like the change is included in branch-2.1-beta, but not branch-2.1.0-beta. > > Particularly since the release notes in RC2 are incorrect in claiming that > this change is in this release, it seems like a pretty serious > issue. Ordinarily I'd say that this issue should result in a new RC, and I > would vote -1 on RC2. But, given the previous discussion that folks are > interested in releasing 2.1.0-beta with several fairly substantial bugs > that we already know about, I'll withhold my vote. If RC2 ends up getting > released as-is, we should be sure to change the fix version field on that > JIRA to be correct. > > -- > Aaron T. Myers > Software Engineer, Cloudera > > > On Thu, Aug 15, 2013 at 2:15 PM, Arun C Murthy wrote: > >> Folks, >> >> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would >> like to get released - this fixes the bugs we saw since the last go-around >> (rc1). >> >> The RC is available at: >> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ >> The RC tag in svn is here: >> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 >> >> The maven artifacts are available via repository.apache.org. >> >> Please try the release and vote; the vote will run for the usual 7 days. >> >> thanks, >> Arun >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)
+1 (binding) Verified bits and ran examples on a 10-node cluster. Looks good. Arun On Aug 15, 2013, at 10:29 PM, Konstantin Boudnik wrote: > All, > > I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would > like to release. > > This is a stabilization release that includes fixed for a couple a of issues > as outlined on the security list. > > The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/ > The RC tag in svn is here: > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1 > > The maven artifacts are available via repository.apache.org. > > The only difference between rc0 and rc1 is ASL added to releasenotes.html and > updated release dates in CHANGES.txt files. > > Please try the release bits and vote; the vote will run for the usual 7 days. > > Thanks for your voting > Cos > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HADOOP-9892) Adding documentation of RPC calls to Hadoop
Elazar Leibovich created HADOOP-9892: Summary: Adding documentation of RPC calls to Hadoop Key: HADOOP-9892 URL: https://issues.apache.org/jira/browse/HADOOP-9892 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Elazar Leibovich Priority: Minor I've written a short document describing the usage and underpinning of Hadoop's RPC mechanism. I'd like to add it to the Hadoop project. http://hadoop.quora.com/Hadoop-RPC-mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
