[jira] [Commented] (HADOOP-9218) Document the Rpc-wrappers used internally
[ https://issues.apache.org/jira/browse/HADOOP-9218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579088#comment-13579088 ] Hudson commented on HADOOP-9218: Integrated in Hadoop-Yarn-trunk #128 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/128/]) HADOOP-9218 Document the Rpc-wrappers used internally (sanjay Radia) (Revision 1446428) Result = SUCCESS sradia : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446428 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java > Document the Rpc-wrappers used internally > - > > Key: HADOOP-9218 > URL: https://issues.apache.org/jira/browse/HADOOP-9218 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: hadoop-9218.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9154) SortedMapWritable#putAll() doesn't add key/value classes to the map
[
https://issues.apache.org/jira/browse/HADOOP-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579089#comment-13579089
]
Hudson commented on HADOOP-9154:
Integrated in Hadoop-Yarn-trunk #128 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/128/])
HADOOP-9154. SortedMapWritable#putAll() doesn't add key/value classes to
the map. Contributed by Karthik Kambatla. (Revision 1446183)
Result = SUCCESS
tomwhite :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446183
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/AbstractMapWritable.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SortedMapWritable.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSortedMapWritable.java
> SortedMapWritable#putAll() doesn't add key/value classes to the map
> ---
>
> Key: HADOOP-9154
> URL: https://issues.apache.org/jira/browse/HADOOP-9154
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
>Affects Versions: 1.1.1, 2.0.2-alpha
>Reporter: Karthik Kambatla
>Assignee: Karthik Kambatla
> Fix For: 1.2.0, 0.23.7, 2.0.4-beta
>
> Attachments: HADOOP-9124.patch, hadoop-9154-branch1.patch,
> hadoop-9154-draft.patch, hadoop-9154-draft.patch, hadoop-9154.patch,
> hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch
>
>
> In the following code from {{SortedMapWritable}}, #putAll() doesn't add
> key/value classes to the class-id maps.
> {code}
> @Override
> public Writable put(WritableComparable key, Writable value) {
> addToMap(key.getClass());
> addToMap(value.getClass());
> return instance.put(key, value);
> }
> @Override
> public void putAll(Map t){
> for (Map.Entry e:
> t.entrySet()) {
>
> instance.put(e.getKey(), e.getValue());
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9304) remove addition of avro genreated-sources dirs to build
[ https://issues.apache.org/jira/browse/HADOOP-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579090#comment-13579090 ] Hudson commented on HADOOP-9304: Integrated in Hadoop-Yarn-trunk #128 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/128/]) HADOOP-9304. remove addition of avro genreated-sources dirs to build. (tucu) (Revision 1446296) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446296 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml > remove addition of avro genreated-sources dirs to build > --- > > Key: HADOOP-9304 > URL: https://issues.apache.org/jira/browse/HADOOP-9304 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 2.0.4-beta >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur > Fix For: 2.0.4-beta > > Attachments: HADOOP-9304.patch > > > The avro maven plugin automatically adds those dirs to the source dirs of the > module. > this is just a POM clean up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9308) Typo in javadoc for IdentityMapper class
[ https://issues.apache.org/jira/browse/HADOOP-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579094#comment-13579094 ] Suresh Srinivas commented on HADOOP-9308: - Can you please file and ICLA. I will add you as a contributor, assign this jira to you and commit this patch. > Typo in javadoc for IdentityMapper class > > > Key: HADOOP-9308 > URL: https://issues.apache.org/jira/browse/HADOOP-9308 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Adam Monsen > Attachments: HADOOP-9308.patch > > > IdentityMapper.map() is incorrectly documented as the "identify" function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9154) SortedMapWritable#putAll() doesn't add key/value classes to the map
[
https://issues.apache.org/jira/browse/HADOOP-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579136#comment-13579136
]
Hudson commented on HADOOP-9154:
Integrated in Hadoop-Hdfs-0.23-Build #526 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/526/])
HADOOP-9154. SortedMapWritable#putAll() doesn't add key/value classes to
the map. (Karthik Kambatla via tgraves) (Revision 1446271)
Result = SUCCESS
tgraves :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446271
Files :
*
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/AbstractMapWritable.java
*
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SortedMapWritable.java
*
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSortedMapWritable.java
> SortedMapWritable#putAll() doesn't add key/value classes to the map
> ---
>
> Key: HADOOP-9154
> URL: https://issues.apache.org/jira/browse/HADOOP-9154
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
>Affects Versions: 1.1.1, 2.0.2-alpha
>Reporter: Karthik Kambatla
>Assignee: Karthik Kambatla
> Fix For: 1.2.0, 0.23.7, 2.0.4-beta
>
> Attachments: HADOOP-9124.patch, hadoop-9154-branch1.patch,
> hadoop-9154-draft.patch, hadoop-9154-draft.patch, hadoop-9154.patch,
> hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch
>
>
> In the following code from {{SortedMapWritable}}, #putAll() doesn't add
> key/value classes to the class-id maps.
> {code}
> @Override
> public Writable put(WritableComparable key, Writable value) {
> addToMap(key.getClass());
> addToMap(value.getClass());
> return instance.put(key, value);
> }
> @Override
> public void putAll(Map t){
> for (Map.Entry e:
> t.entrySet()) {
>
> instance.put(e.getKey(), e.getValue());
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9301) hadoop client servlet/jsp/jetty/tomcat JARs creating conflicts in Oozie & HttpFS
[
https://issues.apache.org/jira/browse/HADOOP-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HADOOP-9301:
--
Fix Version/s: (was: 2.0.3-alpha)
2.0.4-beta
> hadoop client servlet/jsp/jetty/tomcat JARs creating conflicts in Oozie &
> HttpFS
>
>
> Key: HADOOP-9301
> URL: https://issues.apache.org/jira/browse/HADOOP-9301
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.3-alpha
>Reporter: Roman Shaposhnik
>Assignee: Alejandro Abdelnur
>Priority: Blocker
> Fix For: 2.0.4-beta
>
>
> Here's how to reproduce:
> {noformat}
> $ cd hadoop-client
> $ mvn dependency:tree | egrep 'jsp|jetty'
> [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26.cloudera.2:compile
> [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.2:compile
> [INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:compile
> {noformat}
> This has a potential for completely screwing up clients like Oozie, etc –
> hence a blocker.
> It seems that while common excludes those JARs, they are sneaking in via
> hdfs, we need to exclude them too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9125) LdapGroupsMapping threw CommunicationException after some idle time
[ https://issues.apache.org/jira/browse/HADOOP-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-9125: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > LdapGroupsMapping threw CommunicationException after some idle time > --- > > Key: HADOOP-9125 > URL: https://issues.apache.org/jira/browse/HADOOP-9125 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Kai Zheng > Fix For: 2.0.4-beta > > Attachments: HADOOP-9125.patch, HADOOP-9125.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LdapGroupsMapping threw exception as below after some idle time. During the > idle time no call to the group mapping provider should be made to repeat it. > 2012-12-07 02:20:59,738 WARN org.apache.hadoop.security.LdapGroupsMapping: > Exception trying to get groups for user aduser2 > javax.naming.CommunicationException: connection closed [Root exception is > java.io.IOException: connection closed]; remaining name > 'CN=Users,DC=EXAMPLE,DC=COM' > at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1983) > at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1827) > at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1752) > at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1769) > at > com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:394) > at > com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:376) > at > com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358) > at > javax.naming.directory.InitialDirContext.search(InitialDirContext.java:267) > at > org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:187) > at > org.apache.hadoop.security.CompositeGroupsMapping.getGroups(CompositeGroupsMapping.java:97) > at org.apache.hadoop.security.Groups.doGetGroups(Groups.java:103) > at org.apache.hadoop.security.Groups.getGroups(Groups.java:70) > at > org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1035) > at org.apache.hadoop.hbase.security.User.getGroupNames(User.java:90) > at > org.apache.hadoop.hbase.security.access.TableAuthManager.authorize(TableAuthManager.java:355) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:379) > at > org.apache.hadoop.hbase.security.access.AccessController.getUserPermissions(AccessController.java:1051) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:4914) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3546) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:372) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1399) > Caused by: java.io.IOException: connection closed > at com.sun.jndi.ldap.LdapClient.ensureOpen(LdapClient.java:1558) > at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:503) > at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1965) > ... 28 more > 2012-12-07 02:20:59,739 WARN org.apache.hadoop.security.UserGroupInformation: > No groups available for user aduser2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9085) start namenode failure,bacause pid of namenode pid file is other process pid or thread id before start namenode
[ https://issues.apache.org/jira/browse/HADOOP-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-9085: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > start namenode failure,bacause pid of namenode pid file is other process pid > or thread id before start namenode > --- > > Key: HADOOP-9085 > URL: https://issues.apache.org/jira/browse/HADOOP-9085 > Project: Hadoop Common > Issue Type: Bug > Components: bin >Affects Versions: 2.0.1-alpha, 2.0.3-alpha > Environment: NA >Reporter: liaowenrui > Fix For: 2.0.1-alpha, 2.0.2-alpha, 2.0.4-beta > > > pid of namenode pid file is other process pid or thread id before start > namenode,start namenode will failure.because the pid of namenode pid file > will be checked use kill -0 command before start namenode in hadoop-daemo.sh > script.when pid of namenode pid file is other process pid or thread id,checkt > is use kil -0 command,and the kill -0 will return success.it means the > namenode is runing.in really,namenode is not runing. > 2338 is dead namenode pid > 2305 is datanode pid > cqn2:/tmp # kill -0 2338 > cqn2:/tmp # ps -wweLo pid,ppid,tid | grep 2338 > 2305 1 2338 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8943) Support multiple group mapping providers
[ https://issues.apache.org/jira/browse/HADOOP-8943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8943: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > Support multiple group mapping providers > > > Key: HADOOP-8943 > URL: https://issues.apache.org/jira/browse/HADOOP-8943 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Kai Zheng > Fix For: 2.0.4-beta > > Attachments: HADOOP-8943.patch, HADOOP-8943.patch, HADOOP-8943.patch > > Original Estimate: 504h > Remaining Estimate: 504h > > Discussed with Natty about LdapGroupMapping, we need to improve it so that: > 1. It's possible to do different group mapping for different > users/principals. For example, AD user should go to LdapGroupMapping service > for group, but service principals such as hdfs, mapred can still use the > default one ShellBasedUnixGroupsMapping; > 2. Multiple ADs can be supported to do LdapGroupMapping; > 3. It's possible to configure what kind of users/principals (regarding > domain/realm is an option) should use which group mapping service/mechanism. > 4. It's possible to configure and combine multiple existing mapping providers > without writing codes implementing new one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8937) ClientProtocol should support a way to get DataNodeInfo for a particular data node.
[ https://issues.apache.org/jira/browse/HADOOP-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8937: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > ClientProtocol should support a way to get DataNodeInfo for a particular data > node. > --- > > Key: HADOOP-8937 > URL: https://issues.apache.org/jira/browse/HADOOP-8937 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 1.0.3 >Reporter: Sameer Vaishampayan >Priority: Minor > Fix For: 2.0.4-beta > > > HBase project needs a way to find if a DataNode is running local or not on a > given host. The current way is too expensive in which getDatanodeReport needs > to be called which returns information for all data nodes in the cluster. > https://issues.apache.org/jira/browse/HBASE-6398 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the native code using CMake
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8887: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > Use a Maven plugin to build the native code using CMake > --- > > Key: HADOOP-8887 > URL: https://issues.apache.org/jira/browse/HADOOP-8887 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.0.3-alpha >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe >Priority: Minor > Fix For: 2.0.4-beta > > Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, > HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, > HADOOP-8887.006.patch, HADOOP-8887.008.patch, HADOOP-8887.011.patch > > > Currently, we build the native code using ant-build invocations. Although > this works, it has some limitations: > * compiler warning messages are hidden, which can cause people to check in > code with warnings unintentionally > * there is no framework for running native unit tests; instead, we use ad-hoc > constructs involving shell scripts > * the antrun code is very platform specific > * there is no way to run a specific native unit test > * it's more or less impossible for scripts like test-patch.sh to separate a > native test failing from the build itself failing (no files are created) or > to enumerate which native tests failed. > Using a native Maven plugin would overcome these limitations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8884) DEBUG should be WARN for DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError
[ https://issues.apache.org/jira/browse/HADOOP-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8884: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > DEBUG should be WARN for DEBUG util.NativeCodeLoader: Failed to load > native-hadoop with error: java.lang.UnsatisfiedLinkError > - > > Key: HADOOP-8884 > URL: https://issues.apache.org/jira/browse/HADOOP-8884 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.0.1-alpha >Reporter: Anthony Rojas >Assignee: Anthony Rojas > Fix For: 2.0.4-beta > > Attachments: HADOOP-8884.patch, HADOOP-8884.patch, > HADOOP-8884-v2.patch > > > Recommending to change the following debug message and promote it to a > warning instead: > 12/07/02 18:41:44 DEBUG util.NativeCodeLoader: Failed to load native-hadoop > with error: java.lang.UnsatisfiedLinkError: > /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version > `GLIBC_2.6' not found (required by > /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8810) when building with documentation build stops waiting for ENTER on terminal
[
https://issues.apache.org/jira/browse/HADOOP-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HADOOP-8810:
--
Fix Version/s: (was: 2.0.3-alpha)
2.0.4-beta
> when building with documentation build stops waiting for ENTER on terminal
> --
>
> Key: HADOOP-8810
> URL: https://issues.apache.org/jira/browse/HADOOP-8810
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 2.0.3-alpha
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
> Fix For: 2.0.4-beta
>
> Attachments: HADOOP-8810.patch
>
>
> When building the docs {{mvn clean package -Pdocs -DskipTests site site:stage
> -DstagingDirectory=/tmp/hadoop-site}}, in OSX (and I've seen it a few times
> in Ubuntu as well), the build stops, if you press ENTER it continues. It
> happens twice.
> I've traced this down to the exec-maven-plugin invocation of protoc for
> hadoop-yarn-api module (and other YARN module I don't recall at the moment).
> jstacking the Maven process it seems the exec-maven-plugin has some locking
> issues consuming the STDOUT/STDERR of the process being executed.
> I've converted the protoc invocation in the hadoop-yarn-api to use the antrun
> plugin instead and then another module running protoc using exec-maven-puglin
> hang.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8805) Move protocol buffer implementation of GetUserMappingProtocol from HDFS to Common
[ https://issues.apache.org/jira/browse/HADOOP-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8805: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > Move protocol buffer implementation of GetUserMappingProtocol from HDFS to > Common > - > > Key: HADOOP-8805 > URL: https://issues.apache.org/jira/browse/HADOOP-8805 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 2.0.4-beta > > Attachments: HADOOP-8805.patch, HADOOP-8805-v2.patch, > HADOOP-8805-v3.patch > > > org.apache.hadoop.tools.GetUserMappingProtocol is used in both HDFS and YARN. > We should move the protocol buffer implementation from HDFS to Common so that > it can also be used by YARN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8738) junit JAR is showing up in the distro
[ https://issues.apache.org/jira/browse/HADOOP-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8738: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > junit JAR is showing up in the distro > - > > Key: HADOOP-8738 > URL: https://issues.apache.org/jira/browse/HADOOP-8738 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 2.0.2-alpha >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur >Priority: Critical > Fix For: 2.0.4-beta > > Attachments: HADOOP-8738.patch > > > It seems that with the move of YARN module to trunk/ level the test scope in > junit got lost. This makes the junit JAR to show up in the TAR -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8643) hadoop-client should exclude hadoop-annotations from hadoop-common dependency
[ https://issues.apache.org/jira/browse/HADOOP-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8643: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > hadoop-client should exclude hadoop-annotations from hadoop-common dependency > - > > Key: HADOOP-8643 > URL: https://issues.apache.org/jira/browse/HADOOP-8643 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 2.0.2-alpha >Reporter: Alejandro Abdelnur >Priority: Minor > Fix For: 2.0.4-beta > > Attachments: hadoop-8643.txt > > > When reviewing HADOOP-8370 I've missed that changing the scope to compile for > hadoop-annotations in hadoop-common it would make hadoop-annotations to > bubble up in hadoop-client. Because of this we need to explicitly exclude it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8631) The description of net.topology.table.file.name in core-default.xml is misleading
[ https://issues.apache.org/jira/browse/HADOOP-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8631: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > The description of net.topology.table.file.name in core-default.xml is > misleading > - > > Key: HADOOP-8631 > URL: https://issues.apache.org/jira/browse/HADOOP-8631 > Project: Hadoop Common > Issue Type: Bug > Components: conf >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha >Reporter: Han Xiao >Priority: Minor > Fix For: 2.0.4-beta > > Attachments: core-default.xml.patch > > > The net.topology.table.file.name is used when > net.topology.node.switch.mapping.impl property is set to > org.apache.hadoop.net.TableMapping. > However, in the description in core-default.xml, > net.topology.script.file.name property is asked to set to > org.apache.hadoop.net.TableMapping. > This could mislead user into wrong configuration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8500) Javadoc jars contain entire target directory
[ https://issues.apache.org/jira/browse/HADOOP-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8500: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > Javadoc jars contain entire target directory > > > Key: HADOOP-8500 > URL: https://issues.apache.org/jira/browse/HADOOP-8500 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 2.0.0-alpha > Environment: N/A >Reporter: EJ Ciramella >Priority: Minor > Fix For: 2.0.4-beta > > Attachments: HADOOP-8500.patch, site-redo.tar > > Original Estimate: 24h > Remaining Estimate: 24h > > The javadoc jars contain the contents of the target directory - which > includes classes and all sorts of binary files that it shouldn't. > Sometimes the resulting javadoc jar is 10X bigger than it should be. > The fix is to reconfigure maven to use "api" as it's destDir for javadoc > generation. > I have a patch/diff incoming. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8369) Failing tests in branch-2
[ https://issues.apache.org/jira/browse/HADOOP-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8369: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > Failing tests in branch-2 > - > > Key: HADOOP-8369 > URL: https://issues.apache.org/jira/browse/HADOOP-8369 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.0.0-alpha >Reporter: Arun C Murthy > Fix For: 2.0.4-beta > > > Running org.apache.hadoop.io.compress.TestCodec > Tests run: 20, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 27.789 sec > <<< FAILURE! > -- > Running org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem > Tests run: 98, Failures: 0, Errors: 98, Skipped: 0, Time elapsed: 1.633 sec > <<< FAILURE! > -- > Running org.apache.hadoop.fs.viewfs.TestViewFsTrash > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.658 sec <<< > FAILURE! > > TestCodec failed since I didn't pass -Pnative, the test could be improved to > ensure snappy tests are skipped if native hadoop isn't present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8345) HttpServer adds SPNEGO filter mapping but does not register the SPNEGO filter
[ https://issues.apache.org/jira/browse/HADOOP-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8345: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > HttpServer adds SPNEGO filter mapping but does not register the SPNEGO filter > - > > Key: HADOOP-8345 > URL: https://issues.apache.org/jira/browse/HADOOP-8345 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha >Reporter: Alejandro Abdelnur >Priority: Critical > Fix For: 2.0.4-beta > > > It seems the mapping was added to fullfil HDFS requirements, where the SPNEGO > filter is registered. > The registration o the SPNEGO filter should be done at common level instead > to it is avail for all components using HttpServer if security is ON. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8160) HardLink.getLinkCount() is getting stuck in eclipse ( Cygwin) for long file names, due to MS-Dos style Path.
[ https://issues.apache.org/jira/browse/HADOOP-8160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-8160: -- Fix Version/s: (was: 2.0.3-alpha) 2.0.4-beta > HardLink.getLinkCount() is getting stuck in eclipse ( Cygwin) for long file > names, due to MS-Dos style Path. > > > Key: HADOOP-8160 > URL: https://issues.apache.org/jira/browse/HADOOP-8160 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 0.23.1, 0.24.0 > Environment: Cygwin >Reporter: Vinay >Assignee: Vinay >Priority: Minor > Fix For: 3.0.0, 2.0.4-beta > > Attachments: HADOOP-8160.patch > > Original Estimate: 2m > Remaining Estimate: 2m > > HardLink.getLinkCount() is getting stuck in cygwin for long file names, due > to MS-DOS style path. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9218) Document the Rpc-wrappers used internally
[ https://issues.apache.org/jira/browse/HADOOP-9218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579148#comment-13579148 ] Hudson commented on HADOOP-9218: Integrated in Hadoop-Hdfs-trunk #1317 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1317/]) HADOOP-9218 Document the Rpc-wrappers used internally (sanjay Radia) (Revision 1446428) Result = FAILURE sradia : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446428 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java > Document the Rpc-wrappers used internally > - > > Key: HADOOP-9218 > URL: https://issues.apache.org/jira/browse/HADOOP-9218 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: hadoop-9218.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9154) SortedMapWritable#putAll() doesn't add key/value classes to the map
[
https://issues.apache.org/jira/browse/HADOOP-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579149#comment-13579149
]
Hudson commented on HADOOP-9154:
Integrated in Hadoop-Hdfs-trunk #1317 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1317/])
HADOOP-9154. SortedMapWritable#putAll() doesn't add key/value classes to
the map. Contributed by Karthik Kambatla. (Revision 1446183)
Result = FAILURE
tomwhite :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446183
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/AbstractMapWritable.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SortedMapWritable.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSortedMapWritable.java
> SortedMapWritable#putAll() doesn't add key/value classes to the map
> ---
>
> Key: HADOOP-9154
> URL: https://issues.apache.org/jira/browse/HADOOP-9154
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
>Affects Versions: 1.1.1, 2.0.2-alpha
>Reporter: Karthik Kambatla
>Assignee: Karthik Kambatla
> Fix For: 1.2.0, 0.23.7, 2.0.4-beta
>
> Attachments: HADOOP-9124.patch, hadoop-9154-branch1.patch,
> hadoop-9154-draft.patch, hadoop-9154-draft.patch, hadoop-9154.patch,
> hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch
>
>
> In the following code from {{SortedMapWritable}}, #putAll() doesn't add
> key/value classes to the class-id maps.
> {code}
> @Override
> public Writable put(WritableComparable key, Writable value) {
> addToMap(key.getClass());
> addToMap(value.getClass());
> return instance.put(key, value);
> }
> @Override
> public void putAll(Map t){
> for (Map.Entry e:
> t.entrySet()) {
>
> instance.put(e.getKey(), e.getValue());
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9304) remove addition of avro genreated-sources dirs to build
[ https://issues.apache.org/jira/browse/HADOOP-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579150#comment-13579150 ] Hudson commented on HADOOP-9304: Integrated in Hadoop-Hdfs-trunk #1317 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1317/]) HADOOP-9304. remove addition of avro genreated-sources dirs to build. (tucu) (Revision 1446296) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446296 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml > remove addition of avro genreated-sources dirs to build > --- > > Key: HADOOP-9304 > URL: https://issues.apache.org/jira/browse/HADOOP-9304 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 2.0.4-beta >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur > Fix For: 2.0.4-beta > > Attachments: HADOOP-9304.patch > > > The avro maven plugin automatically adds those dirs to the source dirs of the > module. > this is just a POM clean up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9298) Cover with unit test package org.apache.hadoop.hdfs.tools
[ https://issues.apache.org/jira/browse/HADOOP-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Bondarev updated HADOOP-9298: --- Attachment: (was: HADOOP-9298-trunk-a.patch) > Cover with unit test package org.apache.hadoop.hdfs.tools > - > > Key: HADOOP-9298 > URL: https://issues.apache.org/jira/browse/HADOOP-9298 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 >Reporter: Vadim Bondarev > Attachments: HADOOP-9298-branch-0.23-a.patch, > HADOOP-9298-branch-2-a.patch, HADOOP-9298-trunk-a.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9298) Cover with unit test package org.apache.hadoop.hdfs.tools
[ https://issues.apache.org/jira/browse/HADOOP-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Bondarev updated HADOOP-9298: --- Attachment: (was: HADOOP-9298-branch-0.23-a.patch) > Cover with unit test package org.apache.hadoop.hdfs.tools > - > > Key: HADOOP-9298 > URL: https://issues.apache.org/jira/browse/HADOOP-9298 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 >Reporter: Vadim Bondarev > Attachments: HADOOP-9298-branch-0.23-a.patch, > HADOOP-9298-branch-2-a.patch, HADOOP-9298-trunk-a.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9298) Cover with unit test package org.apache.hadoop.hdfs.tools
[ https://issues.apache.org/jira/browse/HADOOP-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Bondarev updated HADOOP-9298: --- Attachment: (was: HADOOP-9298-branch-2-a.patch) > Cover with unit test package org.apache.hadoop.hdfs.tools > - > > Key: HADOOP-9298 > URL: https://issues.apache.org/jira/browse/HADOOP-9298 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 >Reporter: Vadim Bondarev > Attachments: HADOOP-9298-branch-0.23-a.patch, > HADOOP-9298-branch-2-a.patch, HADOOP-9298-trunk-a.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9298) Cover with unit test package org.apache.hadoop.hdfs.tools
[ https://issues.apache.org/jira/browse/HADOOP-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Bondarev updated HADOOP-9298: --- Attachment: HADOOP-9298-trunk-a.patch HADOOP-9298-branch-2-a.patch HADOOP-9298-branch-0.23-a.patch > Cover with unit test package org.apache.hadoop.hdfs.tools > - > > Key: HADOOP-9298 > URL: https://issues.apache.org/jira/browse/HADOOP-9298 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 >Reporter: Vadim Bondarev > Attachments: HADOOP-9298-branch-0.23-a.patch, > HADOOP-9298-branch-2-a.patch, HADOOP-9298-trunk-a.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9310) {FileSystem,FileContext}.getFileBlockLocations() are incompletely and incorrectly specified
Steve Loughran created HADOOP-9310:
--
Summary: {FileSystem,FileContext}.getFileBlockLocations() are
incompletely and incorrectly specified
Key: HADOOP-9310
URL: https://issues.apache.org/jira/browse/HADOOP-9310
Project: Hadoop Common
Issue Type: Bug
Components: fs
Reporter: Steve Loughran
The javadocs of the {{getFileBlockLocations()}} methods in {{FileSystem}} and
{{FileContext}} incorrectly specify the result when a nonexistent region is
passed in (it's an empty array, not null), and nowhere is the response to a
call of the method against a directory defined.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9310) {FileSystem,FileContext}.getFileBlockLocations() are incompletely and incorrectly specified
[
https://issues.apache.org/jira/browse/HADOOP-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579191#comment-13579191
]
Steve Loughran commented on HADOOP-9310:
To fix this, I propose
# the javadocs for the out of range -> empty array policy is defined. The test
{{TestGetFileBlockLocations}} does implicitly test for this against the default
FS, but it isn't part of the contract test suite.
# The expected behaviour for a directory argument needs to be defined, an FS
Contract test written, and all filesystems for which there is built in support
must pass the test.
> {FileSystem,FileContext}.getFileBlockLocations() are incompletely and
> incorrectly specified
> ---
>
> Key: HADOOP-9310
> URL: https://issues.apache.org/jira/browse/HADOOP-9310
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Steve Loughran
>
> The javadocs of the {{getFileBlockLocations()}} methods in {{FileSystem}} and
> {{FileContext}} incorrectly specify the result when a nonexistent region is
> passed in (it's an empty array, not null), and nowhere is the response to a
> call of the method against a directory defined.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9218) Document the Rpc-wrappers used internally
[ https://issues.apache.org/jira/browse/HADOOP-9218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579192#comment-13579192 ] Hudson commented on HADOOP-9218: Integrated in Hadoop-Mapreduce-trunk #1345 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1345/]) HADOOP-9218 Document the Rpc-wrappers used internally (sanjay Radia) (Revision 1446428) Result = SUCCESS sradia : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446428 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java > Document the Rpc-wrappers used internally > - > > Key: HADOOP-9218 > URL: https://issues.apache.org/jira/browse/HADOOP-9218 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: hadoop-9218.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9154) SortedMapWritable#putAll() doesn't add key/value classes to the map
[
https://issues.apache.org/jira/browse/HADOOP-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579193#comment-13579193
]
Hudson commented on HADOOP-9154:
Integrated in Hadoop-Mapreduce-trunk #1345 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1345/])
HADOOP-9154. SortedMapWritable#putAll() doesn't add key/value classes to
the map. Contributed by Karthik Kambatla. (Revision 1446183)
Result = SUCCESS
tomwhite :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446183
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/AbstractMapWritable.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SortedMapWritable.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSortedMapWritable.java
> SortedMapWritable#putAll() doesn't add key/value classes to the map
> ---
>
> Key: HADOOP-9154
> URL: https://issues.apache.org/jira/browse/HADOOP-9154
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
>Affects Versions: 1.1.1, 2.0.2-alpha
>Reporter: Karthik Kambatla
>Assignee: Karthik Kambatla
> Fix For: 1.2.0, 0.23.7, 2.0.4-beta
>
> Attachments: HADOOP-9124.patch, hadoop-9154-branch1.patch,
> hadoop-9154-draft.patch, hadoop-9154-draft.patch, hadoop-9154.patch,
> hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch, hadoop-9154.patch
>
>
> In the following code from {{SortedMapWritable}}, #putAll() doesn't add
> key/value classes to the class-id maps.
> {code}
> @Override
> public Writable put(WritableComparable key, Writable value) {
> addToMap(key.getClass());
> addToMap(value.getClass());
> return instance.put(key, value);
> }
> @Override
> public void putAll(Map t){
> for (Map.Entry e:
> t.entrySet()) {
>
> instance.put(e.getKey(), e.getValue());
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9304) remove addition of avro genreated-sources dirs to build
[ https://issues.apache.org/jira/browse/HADOOP-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579194#comment-13579194 ] Hudson commented on HADOOP-9304: Integrated in Hadoop-Mapreduce-trunk #1345 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1345/]) HADOOP-9304. remove addition of avro genreated-sources dirs to build. (tucu) (Revision 1446296) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1446296 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml > remove addition of avro genreated-sources dirs to build > --- > > Key: HADOOP-9304 > URL: https://issues.apache.org/jira/browse/HADOOP-9304 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 2.0.4-beta >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur > Fix For: 2.0.4-beta > > Attachments: HADOOP-9304.patch > > > The avro maven plugin automatically adds those dirs to the source dirs of the > module. > this is just a POM clean up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579222#comment-13579222 ] Amir Sanjar commented on HADOOP-8419: - Matt, please do not add this patch to the next release.. It causes regression on IBM S390x running zLinux. > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, > HADOOP-8419-trunk-v2.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8989) hadoop dfs -find feature
[ https://issues.apache.org/jira/browse/HADOOP-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579223#comment-13579223 ] Daryn Sharp commented on HADOOP-8989: - I don't remember, but I think LinkedList was used in signatures because of methods unique to that class. I'd recommend any large sweeping changes be considered on another jira. > hadoop dfs -find feature > > > Key: HADOOP-8989 > URL: https://issues.apache.org/jira/browse/HADOOP-8989 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Marco Nicosia >Assignee: Jonathan Allen > Attachments: HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, > HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch > > > Both sysadmins and users make frequent use of the unix 'find' command, but > Hadoop has no correlate. Without this, users are writing scripts which make > heavy use of hadoop dfs -lsr, and implementing find one-offs. I think hdfs > -lsr is somewhat taxing on the NameNode, and a really slow experience on the > client side. Possibly an in-NameNode find operation would be only a bit more > taxing on the NameNode, but significantly faster from the client's point of > view? > The minimum set of options I can think of which would make a Hadoop find > command generally useful is (in priority order): > * -type (file or directory, for now) > * -atime/-ctime-mtime (... and -creationtime?) (both + and - arguments) > * -print0 (for piping to xargs -0) > * -depth > * -owner/-group (and -nouser/-nogroup) > * -name (allowing for shell pattern, or even regex?) > * -perm > * -size > One possible special case, but could possibly be really cool if it ran from > within the NameNode: > * -delete > The "hadoop dfs -lsr | hadoop dfs -rm" cycle is really, really slow. > Lower priority, some people do use operators, mostly to execute -or searches > such as: > * find / \(-nouser -or -nogroup\) > Finally, I thought I'd include a link to the [Posix spec for > find|http://www.opengroup.org/onlinepubs/009695399/utilities/find.html] -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9298) Cover with unit test package org.apache.hadoop.hdfs.tools
[
https://issues.apache.org/jira/browse/HADOOP-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579243#comment-13579243
]
Hadoop QA commented on HADOOP-9298:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569541/HADOOP-9298-trunk-a.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2196//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2196//console
This message is automatically generated.
> Cover with unit test package org.apache.hadoop.hdfs.tools
> -
>
> Key: HADOOP-9298
> URL: https://issues.apache.org/jira/browse/HADOOP-9298
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6
>Reporter: Vadim Bondarev
> Attachments: HADOOP-9298-branch-0.23-a.patch,
> HADOOP-9298-branch-2-a.patch, HADOOP-9298-trunk-a.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9311) hadoop-auth: Kerberos token expiration results in an error
Joel Firehammer created HADOOP-9311: --- Summary: hadoop-auth: Kerberos token expiration results in an error Key: HADOOP-9311 URL: https://issues.apache.org/jira/browse/HADOOP-9311 Project: Hadoop Common Issue Type: Bug Components: net Affects Versions: 1.1.0, 1.0.4 Reporter: Joel Firehammer When using Kerberos for auth, if a page is left open longer than the token timeout, a subsequent request fails (typically to an error page.) Forcing a refresh will re-establish the session. A minor annoyance, but automatically re-authenticating is friendlier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9310) {FileSystem,FileContext}.getFileBlockLocations() are incompletely and incorrectly specified
[
https://issues.apache.org/jira/browse/HADOOP-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579274#comment-13579274
]
Steve Loughran commented on HADOOP-9310:
The current behaviour from the base class is (unintetionally?) to return an
empty location array on a directory.
This is a consequence of a directory having length 0, so the "gone off the end
of the file" logic kicks in
{code}
if (status.getLen() <= start) {
return new BlockLocation[0];
}
{code}
> {FileSystem,FileContext}.getFileBlockLocations() are incompletely and
> incorrectly specified
> ---
>
> Key: HADOOP-9310
> URL: https://issues.apache.org/jira/browse/HADOOP-9310
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Steve Loughran
>
> The javadocs of the {{getFileBlockLocations()}} methods in {{FileSystem}} and
> {{FileContext}} incorrectly specify the result when a nonexistent region is
> passed in (it's an empty array, not null), and nowhere is the response to a
> call of the method against a directory defined.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9311) hadoop-auth: Kerberos token expiration results in an error
[ https://issues.apache.org/jira/browse/HADOOP-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Firehammer updated HADOOP-9311: Attachment: HADOOP-9311-release-1.0.4.patch Instead of throwing an exception on expiration, nullify the toek to force an authentication > hadoop-auth: Kerberos token expiration results in an error > -- > > Key: HADOOP-9311 > URL: https://issues.apache.org/jira/browse/HADOOP-9311 > Project: Hadoop Common > Issue Type: Bug > Components: net >Affects Versions: 1.0.4, 1.1.0 >Reporter: Joel Firehammer > Attachments: HADOOP-9311-release-1.0.4.patch > > > When using Kerberos for auth, if a page is left open longer than the token > timeout, a subsequent request fails (typically to an error page.) Forcing a > refresh will re-establish the session. > A minor annoyance, but automatically re-authenticating is friendlier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9164) Add version number and/or library file name to native library for easy tracking
[ https://issues.apache.org/jira/browse/HADOOP-9164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579346#comment-13579346 ] Sanjay Radia commented on HADOOP-9164: -- bq. If everyone agrees that we should set the Hadoop release number on the native libraries .. Colin, I think you were the last hold out against that ... lets move forward with that in this jira. > Add version number and/or library file name to native library for easy > tracking > --- > > Key: HADOOP-9164 > URL: https://issues.apache.org/jira/browse/HADOOP-9164 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.0.2-alpha >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Minor > Attachments: HADOOP-9164.v1.patch, HADOOP-9164.v2.patch, > HADOOP-9164.v3.patch, HADOOP-9164.v4.2.patch, HADOOP-9164.v4.patch, > HADOOP-9164.v4.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9311) hadoop-auth: Kerberos token expiration results in an error
[ https://issues.apache.org/jira/browse/HADOOP-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579347#comment-13579347 ] Joel Firehammer commented on HADOOP-9311: - Please code review. > hadoop-auth: Kerberos token expiration results in an error > -- > > Key: HADOOP-9311 > URL: https://issues.apache.org/jira/browse/HADOOP-9311 > Project: Hadoop Common > Issue Type: Bug > Components: net >Affects Versions: 1.0.4, 1.1.0 >Reporter: Joel Firehammer > Attachments: HADOOP-9311-release-1.0.4.patch > > > When using Kerberos for auth, if a page is left open longer than the token > timeout, a subsequent request fails (typically to an error page.) Forcing a > refresh will re-establish the session. > A minor annoyance, but automatically re-authenticating is friendlier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579372#comment-13579372 ] Yu Li commented on HADOOP-8419: --- Hi Matt, from the Husdson auto-comment I cannot locate the reason why integration with hdfs-trunk and mapred-trunk failed, can you give me some hints to resolve the issue? Thanks. Hi Amir, can you tell me more about the regression with detailed error/exception message? I think we should improve the patch rather than leave the known issue open, thanks. > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, > HADOOP-8419-trunk-v2.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9125) LdapGroupsMapping threw CommunicationException after some idle time
[ https://issues.apache.org/jira/browse/HADOOP-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HADOOP-9125: --- Target Version/s: 2.0.4-beta (was: 2.0.3-alpha) Fix Version/s: (was: 2.0.4-beta) > LdapGroupsMapping threw CommunicationException after some idle time > --- > > Key: HADOOP-9125 > URL: https://issues.apache.org/jira/browse/HADOOP-9125 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Kai Zheng > Attachments: HADOOP-9125.patch, HADOOP-9125.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LdapGroupsMapping threw exception as below after some idle time. During the > idle time no call to the group mapping provider should be made to repeat it. > 2012-12-07 02:20:59,738 WARN org.apache.hadoop.security.LdapGroupsMapping: > Exception trying to get groups for user aduser2 > javax.naming.CommunicationException: connection closed [Root exception is > java.io.IOException: connection closed]; remaining name > 'CN=Users,DC=EXAMPLE,DC=COM' > at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1983) > at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1827) > at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1752) > at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1769) > at > com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:394) > at > com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:376) > at > com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358) > at > javax.naming.directory.InitialDirContext.search(InitialDirContext.java:267) > at > org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:187) > at > org.apache.hadoop.security.CompositeGroupsMapping.getGroups(CompositeGroupsMapping.java:97) > at org.apache.hadoop.security.Groups.doGetGroups(Groups.java:103) > at org.apache.hadoop.security.Groups.getGroups(Groups.java:70) > at > org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1035) > at org.apache.hadoop.hbase.security.User.getGroupNames(User.java:90) > at > org.apache.hadoop.hbase.security.access.TableAuthManager.authorize(TableAuthManager.java:355) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:379) > at > org.apache.hadoop.hbase.security.access.AccessController.getUserPermissions(AccessController.java:1051) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:4914) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3546) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:372) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1399) > Caused by: java.io.IOException: connection closed > at com.sun.jndi.ldap.LdapClient.ensureOpen(LdapClient.java:1558) > at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:503) > at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1965) > ... 28 more > 2012-12-07 02:20:59,739 WARN org.apache.hadoop.security.UserGroupInformation: > No groups available for user aduser2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9163) The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra copy
[ https://issues.apache.org/jira/browse/HADOOP-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579449#comment-13579449 ] Sanjay Radia commented on HADOOP-9163: -- Todd, given that we are making other changes that break wire compatibility, I would like to make this change since it keeps things cleaner and consistent with the rpc response header and makes minimizing buffer copies simpler; Devraj has noted above that he is doing the same for HBase rpc. In light of that I am posting a patch for moving the rpc msg out of the header. > The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra > copy > -- > > Key: HADOOP-9163 > URL: https://issues.apache.org/jira/browse/HADOOP-9163 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9163) The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra copy
[ https://issues.apache.org/jira/browse/HADOOP-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay Radia updated HADOOP-9163: - Attachment: Hadoop-9163-2.patch > The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra > copy > -- > > Key: HADOOP-9163 > URL: https://issues.apache.org/jira/browse/HADOOP-9163 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia > Attachments: Hadoop-9163-2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9296) Authenticating users from different realm without a trust relationship
[ https://issues.apache.org/jira/browse/HADOOP-9296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579454#comment-13579454 ] Owen O'Malley commented on HADOOP-9296: --- Allen, this is about removing the need for inter-realm trust between the user realm and the service realm. I assume all of the clusters share the same user and service realms, in which case inter-cluster authentication is fine. Am I missing something that would make this harder? > Authenticating users from different realm without a trust relationship > -- > > Key: HADOOP-9296 > URL: https://issues.apache.org/jira/browse/HADOOP-9296 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Benoy Antony >Assignee: Benoy Antony > Attachments: HADOOP-9296-1.1.patch, multirealm.pdf > > > Hadoop Masters (JobTracker and NameNode) and slaves (Data Node and > TaskTracker) are part of the Hadoop domain, controlled by Hadoop Active > Directory. > The users belong to the CORP domain, controlled by the CORP Active Directory. > In the absence of a one way trust from HADOOP DOMAIN to CORP DOMAIN, how will > Hadoop Servers (JobTracker, NameNode) authenticate CORP users ? > The solution and implementation details are in the attachement -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9287) Parallel testing hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated HADOOP-9287: Status: Patch Available (was: Open) > Parallel testing hadoop-common > -- > > Key: HADOOP-9287 > URL: https://issues.apache.org/jira/browse/HADOOP-9287 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0 >Reporter: Tsuyoshi OZAWA >Assignee: Andrey Klochkov > Attachments: HADOOP-9287.1.patch, HADOOP-9287.patch > > > The maven surefire plugin supports parallel testing feature. By using it, the > tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9308) Typo in javadoc for IdentityMapper class
[ https://issues.apache.org/jira/browse/HADOOP-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579542#comment-13579542 ] Adam Monsen commented on HADOOP-9308: - Sure! I filled out the ICLA, created a detached PGP signature, and sent both to secretary at apache. > Typo in javadoc for IdentityMapper class > > > Key: HADOOP-9308 > URL: https://issues.apache.org/jira/browse/HADOOP-9308 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Adam Monsen > Attachments: HADOOP-9308.patch > > > IdentityMapper.map() is incorrectly documented as the "identify" function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9287) Parallel testing hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated HADOOP-9287: Attachment: HADOOP-9287.patch Overview of changes in the patch: # Many static methods in FileContextTestHelper and FileSystemTestHelper are transformed into instance methods. Clients are updated accordingly. # Fixes in tests which used hard-coded paths for temporary data. These paths were modified to to avoid contention with other tests. # Changes in org.apache.hadoop.ha tests to avoid contention when choosing ports for ZK # A few fixes in hadoop-hdfs, hadoop-tools which are required due to changes in hadoop-common > Parallel testing hadoop-common > -- > > Key: HADOOP-9287 > URL: https://issues.apache.org/jira/browse/HADOOP-9287 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0 >Reporter: Tsuyoshi OZAWA >Assignee: Andrey Klochkov > Attachments: HADOOP-9287.1.patch, HADOOP-9287.patch > > > The maven surefire plugin supports parallel testing feature. By using it, the > tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9312) JniBasedUnixGroupsMapping#getGroupForUser can potentially leak memory
Chris Nauroth created HADOOP-9312: - Summary: JniBasedUnixGroupsMapping#getGroupForUser can potentially leak memory Key: HADOOP-9312 URL: https://issues.apache.org/jira/browse/HADOOP-9312 Project: Hadoop Common Issue Type: Bug Components: native, security Affects Versions: 3.0.0, trunk-win Reporter: Chris Nauroth This method lazily initializes a static variable to contain an empty array of groups. If multiple threads call the method before the variable has been initialized, then there is potential for a race condition. This would cause multiple allocations of the array and leaked memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9312) JniBasedUnixGroupsMapping#getGroupForUser can potentially leak memory
[
https://issues.apache.org/jira/browse/HADOOP-9312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579636#comment-13579636
]
Chris Nauroth commented on HADOOP-9312:
---
This came up during code review for HADOOP-9232, which adds a native
implementation for this method on Windows. It turns out that both the Linux
implementation and the Windows implementation headed for branch-trunk-win have
this problem.
Pasting my comments from HADOOP-9232:
Is the following code not thread-safe?
{code}
...
static jobjectArray emptyGroups = NULL;
...
if (emptyGroups == NULL) {
jobjectArray lEmptyGroups = (jobjectArray)(*env)->NewObjectArray(env, 0,
(*env)->FindClass(env, "java/lang/String"), NULL);
if (lEmptyGroups == NULL) {
goto cleanup;
}
emptyGroups = (*env)->NewGlobalRef(env, lEmptyGroups);
if (emptyGroups == NULL) {
goto cleanup;
}
}
{code}
For example, assume 2 threads concurrently call getGroupForUser. Thread 1
executes the NULL check, enters the if body, and then gets suspended by the OS.
Thread 2 executes and emptyGroups is still NULL, so it initializes it. Then,
the OS resumes thread 1, which proceeds inside the if body and calls
NewObjectArray again. Since emptyGroups never gets freed, I believe the net
effect would be a small memory leak.
> JniBasedUnixGroupsMapping#getGroupForUser can potentially leak memory
> -
>
> Key: HADOOP-9312
> URL: https://issues.apache.org/jira/browse/HADOOP-9312
> Project: Hadoop Common
> Issue Type: Bug
> Components: native, security
>Affects Versions: 3.0.0, trunk-win
>Reporter: Chris Nauroth
>
> This method lazily initializes a static variable to contain an empty array of
> groups. If multiple threads call the method before the variable has been
> initialized, then there is potential for a race condition. This would cause
> multiple allocations of the array and leaked memory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9232) JniBasedUnixGroupsMappingWithFallback fails on Windows with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HADOOP-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579652#comment-13579652
]
Chris Nauroth commented on HADOOP-9232:
---
+1 for the new patch
Thank you for addressing all of the feedback. Regarding #4, you're right that
the memory leak is already there in the Linux implementation too. I filed
HADOOP-9312 to follow up on fixing it for both Linux and Windows.
I ran the new patch locally and confirmed that it works. Thanks again for
taking care of this.
> JniBasedUnixGroupsMappingWithFallback fails on Windows with
> UnsatisfiedLinkError
>
>
> Key: HADOOP-9232
> URL: https://issues.apache.org/jira/browse/HADOOP-9232
> Project: Hadoop Common
> Issue Type: Bug
> Components: native, security
>Affects Versions: trunk-win
>Reporter: Chris Nauroth
>Assignee: Ivan Mitic
> Attachments: HADOOP-9232.branch-trunk-win.jnigroups.2.patch,
> HADOOP-9232.branch-trunk-win.jnigroups.3.patch,
> HADOOP-9232.branch-trunk-win.jnigroups.patch
>
>
> {{JniBasedUnixGroupsMapping}} calls native code which isn't implemented
> properly for Windows, causing {{UnsatisfiedLinkError}}. The fallback logic
> in {{JniBasedUnixGroupsMappingWithFallback}} works by checking if the native
> code is loaded during startup. In this case, hadoop.dll is present and
> loaded, but it doesn't contain the right code. There will be no attempt to
> fallback to {{ShellBasedUnixGroupsMapping}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9287) Parallel testing hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579672#comment-13579672
]
Hadoop QA commented on HADOOP-9287:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12569614/HADOOP-9287.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 52 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs
hadoop-tools/hadoop-distcp
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.tools.mapred.TestUniformSizeInputFormat
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2197//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2197//console
This message is automatically generated.
> Parallel testing hadoop-common
> --
>
> Key: HADOOP-9287
> URL: https://issues.apache.org/jira/browse/HADOOP-9287
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Tsuyoshi OZAWA
>Assignee: Andrey Klochkov
> Attachments: HADOOP-9287.1.patch, HADOOP-9287.patch
>
>
> The maven surefire plugin supports parallel testing feature. By using it, the
> tests can be run more faster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
