[jira] [Commented] (HADOOP-9042) Add a test for umask in FileSystemContractBaseTest
[
https://issues.apache.org/jira/browse/HADOOP-9042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501845#comment-13501845
]
Hudson commented on HADOOP-9042:
Integrated in Hadoop-Yarn-trunk #43 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/43/])
HADOOP-9042. Add a test for umask in FileSystemContractBaseTest.
Contributed by Colin McCabe (Revision 1411879)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411879
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSFileSystemContract.java
Add a test for umask in FileSystemContractBaseTest
--
Key: HADOOP-9042
URL: https://issues.apache.org/jira/browse/HADOOP-9042
Project: Hadoop Common
Issue Type: Test
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Fix For: 2.0.3-alpha
Attachments: HADOOP-9042.008.patch, HADOOP-9042.009.patch,
HADOOP-9042.010.patch, HDFS-3975.001.patch, HDFS-3975.002.patch,
HDFS-3975.003.patch, HDFS-3975.004.patch, HDFS-3975.005.patch,
HDFS-3975.006.patch
Add a unit test to make sure {{umask}} is working correctly in
FileSystemContractBaseTest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9075) FileContext#FSLinkResolver should be made static
[ https://issues.apache.org/jira/browse/HADOOP-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501846#comment-13501846 ] Hudson commented on HADOOP-9075: Integrated in Hadoop-Yarn-trunk #43 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/43/]) HADOOP-9075. FileContext#FSLinkResolver should be made static. Contributed by Arpit Agarwal. (Revision 1411988) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411988 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java FileContext#FSLinkResolver should be made static Key: HADOOP-9075 URL: https://issues.apache.org/jira/browse/HADOOP-9075 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Fix For: 3.0.0 Attachments: HDFS-4218.patch FSLinkResolver class does not require access to Outer Class and hence should be made static. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501848#comment-13501848 ] Hudson commented on HADOOP-9072: Integrated in Hadoop-Yarn-trunk #43 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/43/]) HADOOP-9072. Hadoop-Common-0.23-Build Fails to build in Jenkins (Robert Parker via tgraves) (Revision 1411884) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411884 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/pom.xml Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Fix For: 3.0.0, 2.0.3-alpha, 0.23.6 Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9045) In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup
[ https://issues.apache.org/jira/browse/HADOOP-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HADOOP-9045: --- Attachment: HADOOP-9045-v3.patch Fix findbugs warning in v3 patch In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup -- Key: HADOOP-9045 URL: https://issues.apache.org/jira/browse/HADOOP-9045 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Junping Du Assignee: Junping Du Attachments: HADOOP-9045.patch, HADOOP-9045-v2.patch, HADOOP-9045-v3.patch In previous implementation for HADOOP-8468, 3rd replica is avoid to place on the same nodegroup of 2nd replica. But it didn't provide check on nodegroup of 1st replica, so if 2nd replica's rack is not efficient to place replica, then it is possible to place 3rd and 1st replica within the same node group. We need a change to remove all nodes from available nodes for placing replica if there already replica on the same nodegroup. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9049) DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from.
[ https://issues.apache.org/jira/browse/HADOOP-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501918#comment-13501918 ] Hudson commented on HADOOP-9049: Integrated in Hadoop-trunk-Commit #3053 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3053/]) HADOOP-9049. DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. Contributed by Karthik Kambatla. (Revision 1412077) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1412077 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. Key: HADOOP-9049 URL: https://issues.apache.org/jira/browse/HADOOP-9049 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.0.2-alpha Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: hadoop-9049.patch, hadoop-9049.patch Currently, DelegationTokenRenewer is not singleton. Each filesystem using it spawns its own DelegationTokenRenewer. Also, they don't stop the Renewer leading to other problems. A single DelegationTokenRenewer should be sufficient for all FileSystems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9049) DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from.
[ https://issues.apache.org/jira/browse/HADOOP-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated HADOOP-9049: -- Resolution: Fixed Fix Version/s: 2.0.3-alpha Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I just committed this. Thanks for the patch, Karthik, and for the review, Daryn. DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. Key: HADOOP-9049 URL: https://issues.apache.org/jira/browse/HADOOP-9049 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.0.2-alpha Reporter: Karthik Kambatla Assignee: Karthik Kambatla Fix For: 2.0.3-alpha Attachments: hadoop-9049.patch, hadoop-9049.patch Currently, DelegationTokenRenewer is not singleton. Each filesystem using it spawns its own DelegationTokenRenewer. Also, they don't stop the Renewer leading to other problems. A single DelegationTokenRenewer should be sufficient for all FileSystems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8860) Split MapReduce and YARN sections in documentation navigation
[ https://issues.apache.org/jira/browse/HADOOP-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501920#comment-13501920 ] Hudson commented on HADOOP-8860: Integrated in Hadoop-Hdfs-0.23-Build #442 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/442/]) HADOOP-8860. Split MapReduce and YARN sections in documentation navigation. (Revision 1411762) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411762 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/resources * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/resources/css * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/resources/css/site.css * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/site.xml * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/resources/css * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/resources/css/site.css * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site/apt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/WebHDFS.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site/resources * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/css * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site/resources/css/site.css * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/site/site.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/resources * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/resources/css * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/resources/css/site.css * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/site.xml * /hadoop/common/branches/branch-0.23/hadoop-project/src/site/apt/index.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-project/src/site/site.xml * /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/Federation.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/SingleCluster.apt.vm * /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm Split MapReduce and YARN sections in documentation navigation - Key: HADOOP-8860 URL: https://issues.apache.org/jira/browse/HADOOP-8860 Project: Hadoop Common Issue Type: Task Components: documentation Affects Versions: 2.0.1-alpha Reporter: Tom White Assignee: Tom White Fix For: 2.0.3-alpha, 0.23.5 Attachments: HADOOP-8860-23.patch, HADOOP-8860-23.sh, HADOOP-8860.patch, HADOOP-8860.patch, HADOOP-8860.sh, HADOOP-8860.sh This JIRA is to change the navigation on http://hadoop.apache.org/docs/r2.0.1-alpha/ to reflect the fact that MapReduce and YARN are separate modules/sub-projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501921#comment-13501921 ] Hudson commented on HADOOP-9072: Integrated in Hadoop-Hdfs-0.23-Build #442 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/442/]) merge -r 1411883:1411884 from trunk. FIXES: HADOOP-9072 (Revision 1411892) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411892 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/pom.xml Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Fix For: 3.0.0, 2.0.3-alpha, 0.23.6 Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9045) In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup
[
https://issues.apache.org/jira/browse/HADOOP-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501934#comment-13501934
]
Hadoop QA commented on HADOOP-9045:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554487/HADOOP-9045-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1786//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1786//console
This message is automatically generated.
In nodegroup-aware case, make sure nodes are avoided to place replica if some
replica are already under the same nodegroup
--
Key: HADOOP-9045
URL: https://issues.apache.org/jira/browse/HADOOP-9045
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.2-alpha
Reporter: Junping Du
Assignee: Junping Du
Attachments: HADOOP-9045.patch, HADOOP-9045-v2.patch,
HADOOP-9045-v3.patch
In previous implementation for HADOOP-8468, 3rd replica is avoid to place on
the same nodegroup of 2nd replica. But it didn't provide check on nodegroup
of 1st replica, so if 2nd replica's rack is not efficient to place replica,
then it is possible to place 3rd and 1st replica within the same node group.
We need a change to remove all nodes from available nodes for placing replica
if there already replica on the same nodegroup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9042) Add a test for umask in FileSystemContractBaseTest
[
https://issues.apache.org/jira/browse/HADOOP-9042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501940#comment-13501940
]
Hudson commented on HADOOP-9042:
Integrated in Hadoop-Hdfs-trunk #1233 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1233/])
HADOOP-9042. Add a test for umask in FileSystemContractBaseTest.
Contributed by Colin McCabe (Revision 1411879)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411879
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSFileSystemContract.java
Add a test for umask in FileSystemContractBaseTest
--
Key: HADOOP-9042
URL: https://issues.apache.org/jira/browse/HADOOP-9042
Project: Hadoop Common
Issue Type: Test
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Fix For: 2.0.3-alpha
Attachments: HADOOP-9042.008.patch, HADOOP-9042.009.patch,
HADOOP-9042.010.patch, HDFS-3975.001.patch, HDFS-3975.002.patch,
HDFS-3975.003.patch, HDFS-3975.004.patch, HDFS-3975.005.patch,
HDFS-3975.006.patch
Add a unit test to make sure {{umask}} is working correctly in
FileSystemContractBaseTest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9075) FileContext#FSLinkResolver should be made static
[ https://issues.apache.org/jira/browse/HADOOP-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501941#comment-13501941 ] Hudson commented on HADOOP-9075: Integrated in Hadoop-Hdfs-trunk #1233 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1233/]) HADOOP-9075. FileContext#FSLinkResolver should be made static. Contributed by Arpit Agarwal. (Revision 1411988) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411988 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java FileContext#FSLinkResolver should be made static Key: HADOOP-9075 URL: https://issues.apache.org/jira/browse/HADOOP-9075 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Fix For: 3.0.0 Attachments: HDFS-4218.patch FSLinkResolver class does not require access to Outer Class and hence should be made static. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501943#comment-13501943 ] Hudson commented on HADOOP-9072: Integrated in Hadoop-Hdfs-trunk #1233 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1233/]) HADOOP-9072. Hadoop-Common-0.23-Build Fails to build in Jenkins (Robert Parker via tgraves) (Revision 1411884) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411884 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/pom.xml Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Fix For: 3.0.0, 2.0.3-alpha, 0.23.6 Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9049) DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from.
[ https://issues.apache.org/jira/browse/HADOOP-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501960#comment-13501960 ] Hudson commented on HADOOP-9049: Integrated in Hadoop-Mapreduce-trunk #1264 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1264/]) HADOOP-9049. DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. Contributed by Karthik Kambatla. (Revision 1412077) Result = FAILURE tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1412077 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. Key: HADOOP-9049 URL: https://issues.apache.org/jira/browse/HADOOP-9049 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.0.2-alpha Reporter: Karthik Kambatla Assignee: Karthik Kambatla Fix For: 2.0.3-alpha Attachments: hadoop-9049.patch, hadoop-9049.patch Currently, DelegationTokenRenewer is not singleton. Each filesystem using it spawns its own DelegationTokenRenewer. Also, they don't stop the Renewer leading to other problems. A single DelegationTokenRenewer should be sufficient for all FileSystems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9075) FileContext#FSLinkResolver should be made static
[ https://issues.apache.org/jira/browse/HADOOP-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501961#comment-13501961 ] Hudson commented on HADOOP-9075: Integrated in Hadoop-Mapreduce-trunk #1264 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1264/]) HADOOP-9075. FileContext#FSLinkResolver should be made static. Contributed by Arpit Agarwal. (Revision 1411988) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411988 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java FileContext#FSLinkResolver should be made static Key: HADOOP-9075 URL: https://issues.apache.org/jira/browse/HADOOP-9075 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Fix For: 3.0.0 Attachments: HDFS-4218.patch FSLinkResolver class does not require access to Outer Class and hence should be made static. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9042) Add a test for umask in FileSystemContractBaseTest
[
https://issues.apache.org/jira/browse/HADOOP-9042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501959#comment-13501959
]
Hudson commented on HADOOP-9042:
Integrated in Hadoop-Mapreduce-trunk #1264 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1264/])
HADOOP-9042. Add a test for umask in FileSystemContractBaseTest.
Contributed by Colin McCabe (Revision 1411879)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411879
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSFileSystemContract.java
Add a test for umask in FileSystemContractBaseTest
--
Key: HADOOP-9042
URL: https://issues.apache.org/jira/browse/HADOOP-9042
Project: Hadoop Common
Issue Type: Test
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Fix For: 2.0.3-alpha
Attachments: HADOOP-9042.008.patch, HADOOP-9042.009.patch,
HADOOP-9042.010.patch, HDFS-3975.001.patch, HDFS-3975.002.patch,
HDFS-3975.003.patch, HDFS-3975.004.patch, HDFS-3975.005.patch,
HDFS-3975.006.patch
Add a unit test to make sure {{umask}} is working correctly in
FileSystemContractBaseTest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501963#comment-13501963 ] Hudson commented on HADOOP-9072: Integrated in Hadoop-Mapreduce-trunk #1264 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1264/]) HADOOP-9072. Hadoop-Common-0.23-Build Fails to build in Jenkins (Robert Parker via tgraves) (Revision 1411884) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411884 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/pom.xml Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Fix For: 3.0.0, 2.0.3-alpha, 0.23.6 Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9045) In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup
[ https://issues.apache.org/jira/browse/HADOOP-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HADOOP-9045: --- Attachment: HADOOP-9045-v4.patch I didn't see the same UT error in local environment, so rename the patch to v4 and submit to run PreCommit test again. In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup -- Key: HADOOP-9045 URL: https://issues.apache.org/jira/browse/HADOOP-9045 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Junping Du Assignee: Junping Du Attachments: HADOOP-9045.patch, HADOOP-9045-v2.patch, HADOOP-9045-v3.patch, HADOOP-9045-v4.patch In previous implementation for HADOOP-8468, 3rd replica is avoid to place on the same nodegroup of 2nd replica. But it didn't provide check on nodegroup of 1st replica, so if 2nd replica's rack is not efficient to place replica, then it is possible to place 3rd and 1st replica within the same node group. We need a change to remove all nodes from available nodes for placing replica if there already replica on the same nodegroup. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9076) Fix QA bot in YARN project
Radim Kolar created HADOOP-9076: --- Summary: Fix QA bot in YARN project Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Task Components: build Reporter: Radim Kolar purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9045) In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup
[
https://issues.apache.org/jira/browse/HADOOP-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502034#comment-13502034
]
Hadoop QA commented on HADOOP-9045:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554497/HADOOP-9045-v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1787//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1787//console
This message is automatically generated.
In nodegroup-aware case, make sure nodes are avoided to place replica if some
replica are already under the same nodegroup
--
Key: HADOOP-9045
URL: https://issues.apache.org/jira/browse/HADOOP-9045
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.2-alpha
Reporter: Junping Du
Assignee: Junping Du
Attachments: HADOOP-9045.patch, HADOOP-9045-v2.patch,
HADOOP-9045-v3.patch, HADOOP-9045-v4.patch
In previous implementation for HADOOP-8468, 3rd replica is avoid to place on
the same nodegroup of 2nd replica. But it didn't provide check on nodegroup
of 1st replica, so if 2nd replica's rack is not efficient to place replica,
then it is possible to place 3rd and 1st replica within the same node group.
We need a change to remove all nodes from available nodes for placing replica
if there already replica on the same nodegroup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: pstree-update2.txt Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Task Components: build Reporter: Radim Kolar Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Priority: Trivial (was: Major) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Task Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9077) Make streaming to output rawbytes and typed bytes only
Miguel Hernandez created HADOOP-9077: Summary: Make streaming to output rawbytes and typed bytes only Key: HADOOP-9077 URL: https://issues.apache.org/jira/browse/HADOOP-9077 Project: Hadoop Common Issue Type: Improvement Reporter: Miguel Hernandez With https://issues.apache.org/jira/browse/HADOOP-1722, ability for typedbytes and rawbytes was added, it actually handles the byte output with the SequenceFileBinaryOutputFormat. By implementing a RawBytesOutputFormat and TypedBytesOutputFormat, the resulting hadoop files could be taken to third party applications which are not written in Java and easily readed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9077) Make streaming to output rawbytes and typed bytes only
[ https://issues.apache.org/jira/browse/HADOOP-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miguel Hernandez updated HADOOP-9077: - Attachment: RawBytesOutputFormat.java Example implementation for raw bytes output format. Make streaming to output rawbytes and typed bytes only -- Key: HADOOP-9077 URL: https://issues.apache.org/jira/browse/HADOOP-9077 Project: Hadoop Common Issue Type: Improvement Reporter: Miguel Hernandez Attachments: RawBytesOutputFormat.java With https://issues.apache.org/jira/browse/HADOOP-1722, ability for typedbytes and rawbytes was added, it actually handles the byte output with the SequenceFileBinaryOutputFormat. By implementing a RawBytesOutputFormat and TypedBytesOutputFormat, the resulting hadoop files could be taken to third party applications which are not written in Java and easily readed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
[ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502060#comment-13502060 ] Yanbo Liang commented on HADOOP-9041: - @Radim, in maven+junit test case may not fail is not caused by reduplicative initialization. Because if reduplicative initialization occur, it will throw Error. As my opinion, it caused by the maven+junit framework has load the implementation of FileSystem before the test case running, so it will call ServiceLoader ahead of registering FsUrlStreamHandlerFactory all the time. So the test case can not reproduce in maven+junit. FileSystem initialization can go into infinite loop --- Key: HADOOP-9041 URL: https://issues.apache.org/jira/browse/HADOOP-9041 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Radim Kolar Assignee: Yanbo Liang Priority: Critical Attachments: fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, HADOOP-9041.patch More information is there: https://jira.springsource.org/browse/SHDP-111 Referenced source code from example is: https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java from isolating that cause it looks like if you register: org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling FileSystem.loadFileSystems() then it goes into infinite loop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502073#comment-13502073 ] Tom White commented on HADOOP-9076: --- Radim - Vinod and I have been trying to get the automated commit jobs working for YARN. Gavin from Apache Infra kindly fixed it yesterday: http://mail-archives.apache.org/mod_mbox/www-builds/201211.mbox/%3Ccb6401cdc764%24421a76a0%24c64f63e0%24%4016degrees.com.au%3E Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Task Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502083#comment-13502083 ] Radim Kolar commented on HADOOP-9076: - i will test it Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Task Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502088#comment-13502088
]
Robert Joseph Evans commented on HADOOP-9070:
-
The new patch looks OK to me. Have you tested this on a secure cluster to be
sure there isn't anything else lurking behind the scenes? Have you checked for
backwards compatibility?
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch, HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Issue Type: Bug (was: Task) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: pstree-update2.txt) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: pstree-update2.txt Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502094#comment-13502094 ] Radim Kolar commented on HADOOP-9076: - i submitted something to YARN but do not see any build activity at https://builds.apache.org/view/Hadoop/ Last pre commit check for YARN is 1 day 22 hours old. Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502098#comment-13502098
]
Daryn Sharp commented on HADOOP-9070:
-
Yes, I thoroughly tested both kerberos and tokens to ensure they work with the
latest patch. As mentioned in my prior comment, this does cause a RPC
incompatibility within 2.x. Earlier 2.x clients will receive an extra reply
(SUCCESS) from a 2.0.3 server after the kerberos negotiation completes. The
client will interpret this as the response for the next proxy call, which will
cause a protobuf error. A 2.0.3 client will timeout waiting for the SUCCESS
response from earlier 2.x servers. Maybe we should bump the RPC version in
2.0.3? Or if that's unpalatable, I can investigate a backwards compatible
client change that might be hacky (not sure yet).
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch, HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9078) enhance unit-test coverage of class org.apache.hadoop.fs.FileContext
Ivan A. Veselovsky created HADOOP-9078: -- Summary: enhance unit-test coverage of class org.apache.hadoop.fs.FileContext Key: HADOOP-9078 URL: https://issues.apache.org/jira/browse/HADOOP-9078 Project: Hadoop Common Issue Type: Test Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502107#comment-13502107 ] Radim Kolar commented on HADOOP-9076: - this is job collecting tickets from JIRA - https://builds.apache.org/job/PreCommit-Admin/93676/console I see no YARN tickets in build queue Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Status: Patch Available (was: Open) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update2.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502126#comment-13502126
]
Owen O'Malley commented on HADOOP-9070:
---
This looks good, but if the versions won't talk to each other we should bump
the rpc version.
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch, HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[
https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502127#comment-13502127
]
Hadoop QA commented on HADOOP-9076:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554519/pstree-update2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1788//console
This message is automatically generated.
Fix QA bot in YARN project
--
Key: HADOOP-9076
URL: https://issues.apache.org/jira/browse/HADOOP-9076
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Radim Kolar
Priority: Trivial
Attachments: pstree-update2.txt
purpose of this ticket is to run automatic tests on patches to YARN project
which have not working QA bot. There seems to be zero interest in fixing QA
bot to make it work for YARN subproject.
If you need to get your patch for YARN tested, just subscribe and attach it
to this ticket.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: pstree-update2.txt) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update3.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: pstree-update3.txt Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update3.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
Jimmy Xiang created HADOOP-9079: --- Summary: LocalDirAllocator throws ArithmeticException Key: HADOOP-9079 URL: https://issues.apache.org/jira/browse/HADOOP-9079 Project: Hadoop Common Issue Type: Bug Reporter: Jimmy Xiang Priority: Minor 2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671] nodemanager.NMAuditLogger(150): USER=UnknownUser IP= OPERATION=Stop Container RequestTARGET=ContainerManagerImpl RESULT=FAILURE DESCRIPTION=Trying to stop unknown container! APPID=application_1353391620476_0001 CONTAINERID=container_1353391620476_0001_01_10 java.lang.ArithmeticException: / by zero at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
[ https://issues.apache.org/jira/browse/HADOOP-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HADOOP-9079: Attachment: trunk-9079.patch LocalDirAllocator throws ArithmeticException Key: HADOOP-9079 URL: https://issues.apache.org/jira/browse/HADOOP-9079 Project: Hadoop Common Issue Type: Bug Reporter: Jimmy Xiang Priority: Minor Attachments: trunk-9079.patch 2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671] nodemanager.NMAuditLogger(150): USER=UnknownUserIP= OPERATION=Stop Container RequestTARGET=ContainerManagerImpl RESULT=FAILURE DESCRIPTION=Trying to stop unknown container! APPID=application_1353391620476_0001 CONTAINERID=container_1353391620476_0001_01_10 java.lang.ArithmeticException: / by zero at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
[ https://issues.apache.org/jira/browse/HADOOP-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HADOOP-9079: Status: Patch Available (was: Open) LocalDirAllocator throws ArithmeticException Key: HADOOP-9079 URL: https://issues.apache.org/jira/browse/HADOOP-9079 Project: Hadoop Common Issue Type: Bug Reporter: Jimmy Xiang Priority: Minor Attachments: trunk-9079.patch 2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671] nodemanager.NMAuditLogger(150): USER=UnknownUserIP= OPERATION=Stop Container RequestTARGET=ContainerManagerImpl RESULT=FAILURE DESCRIPTION=Trying to stop unknown container! APPID=application_1353391620476_0001 CONTAINERID=container_1353391620476_0001_01_10 java.lang.ArithmeticException: / by zero at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[
https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502146#comment-13502146
]
Hadoop QA commented on HADOOP-9076:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554531/pstree-update3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1789//console
This message is automatically generated.
Fix QA bot in YARN project
--
Key: HADOOP-9076
URL: https://issues.apache.org/jira/browse/HADOOP-9076
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Radim Kolar
Priority: Trivial
Attachments: pstree-update3.txt
purpose of this ticket is to run automatic tests on patches to YARN project
which have not working QA bot. There seems to be zero interest in fixing QA
bot to make it work for YARN subproject.
If you need to get your patch for YARN tested, just subscribe and attach it
to this ticket.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
[
https://issues.apache.org/jira/browse/HADOOP-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502150#comment-13502150
]
Eli Collins commented on HADOOP-9079:
-
I assume this happens when the local dirs are out of space? Don't think just
checking totalAvailable 0 fixes this, the following loop doesn't bounds check
dir, needs to be re-written with a test.
{code}
long randomPosition = Math.abs(r.nextLong()) % totalAvailable;
int dir = 0;
while (randomPosition availableOnDisk[dir]) {
randomPosition -= availableOnDisk[dir];
dir++;
}
{code}
LocalDirAllocator throws ArithmeticException
Key: HADOOP-9079
URL: https://issues.apache.org/jira/browse/HADOOP-9079
Project: Hadoop Common
Issue Type: Bug
Reporter: Jimmy Xiang
Priority: Minor
Attachments: trunk-9079.patch
2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671]
nodemanager.NMAuditLogger(150): USER=UnknownUserIP=
OPERATION=Stop Container RequestTARGET=ContainerManagerImpl
RESULT=FAILURE DESCRIPTION=Trying to stop unknown container!
APPID=application_1353391620476_0001
CONTAINERID=container_1353391620476_0001_01_10
java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115)
at
org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9080) Update bylaws to specify PMC chair voting
Eli Collins created HADOOP-9080: --- Summary: Update bylaws to specify PMC chair voting Key: HADOOP-9080 URL: https://issues.apache.org/jira/browse/HADOOP-9080 Project: Hadoop Common Issue Type: Task Reporter: Eli Collins Assignee: Eli Collins The PMC voted to recommend a new chair using STV, let's update the bylaws accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9080) Update bylaws to specify PMC chair voting
[ https://issues.apache.org/jira/browse/HADOOP-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HADOOP-9080: Attachment: hadoop-9080.txt Patch attached. Update bylaws to specify PMC chair voting - Key: HADOOP-9080 URL: https://issues.apache.org/jira/browse/HADOOP-9080 Project: Hadoop Common Issue Type: Task Reporter: Eli Collins Assignee: Eli Collins Attachments: hadoop-9080.txt The PMC voted to recommend a new chair using STV, let's update the bylaws accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
[
https://issues.apache.org/jira/browse/HADOOP-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502160#comment-13502160
]
Hadoop QA commented on HADOOP-9079:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554534/trunk-9079.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1790//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1790//console
This message is automatically generated.
LocalDirAllocator throws ArithmeticException
Key: HADOOP-9079
URL: https://issues.apache.org/jira/browse/HADOOP-9079
Project: Hadoop Common
Issue Type: Bug
Reporter: Jimmy Xiang
Priority: Minor
Attachments: trunk-9079.patch
2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671]
nodemanager.NMAuditLogger(150): USER=UnknownUserIP=
OPERATION=Stop Container RequestTARGET=ContainerManagerImpl
RESULT=FAILURE DESCRIPTION=Trying to stop unknown container!
APPID=application_1353391620476_0001
CONTAINERID=container_1353391620476_0001_01_10
java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115)
at
org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9081) Port TestWinUtils from branch-1-win to branch-trunk-win
Chris Nauroth created HADOOP-9081: - Summary: Port TestWinUtils from branch-1-win to branch-trunk-win Key: HADOOP-9081 URL: https://issues.apache.org/jira/browse/HADOOP-9081 Project: Hadoop Common Issue Type: Improvement Components: test Affects Versions: trunk-win Reporter: Chris Nauroth Assignee: Chris Nauroth branch-1-win has a test suite named TestWinUtils to cover winutils.exe functionality. We need to port this to branch-trunk-win. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
[ https://issues.apache.org/jira/browse/HADOOP-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502171#comment-13502171 ] Jimmy Xiang commented on HADOOP-9079: - @Eli, could you add me as a Hadoop contributor? This could also happen when the local dirs are not writable. The second while loop seems to be bounded implicitly since randomPosition totalAvailable and sum of (availableOnDisk[dir]) = totalAvailable. Let me think it more. Sure, will add a test. LocalDirAllocator throws ArithmeticException Key: HADOOP-9079 URL: https://issues.apache.org/jira/browse/HADOOP-9079 Project: Hadoop Common Issue Type: Bug Reporter: Jimmy Xiang Priority: Minor Attachments: trunk-9079.patch 2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671] nodemanager.NMAuditLogger(150): USER=UnknownUserIP= OPERATION=Stop Container RequestTARGET=ContainerManagerImpl RESULT=FAILURE DESCRIPTION=Trying to stop unknown container! APPID=application_1353391620476_0001 CONTAINERID=container_1353391620476_0001_01_10 java.lang.ArithmeticException: / by zero at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502172#comment-13502172 ] Chris Nauroth commented on HADOOP-9073: --- Hello, Radim. I just want to confirm: is this a merge to trunk of changes HADOOP-8924 and HADOOP-9011? Thank you. Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0 Reporter: Radim Kolar Attachments: version-py2.txt, version-py3.txt, version-py.txt Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502179#comment-13502179 ] Radim Kolar commented on HADOOP-9073: - No, i did similar fix to HADOOP-9011 myself and also added support for svn. Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0 Reporter: Radim Kolar Attachments: version-py2.txt, version-py3.txt, version-py.txt Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: pstree-update4.txt Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update3.txt, pstree-update4.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: pstree-update3.txt) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update4.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment
Matt Foley created HADOOP-9082: -- Summary: Select and document a platform-independent scripting language for use in Hadoop environment Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082 Project: Hadoop Common Issue Type: Bug Reporter: Matt Foley This issue is going to be discussed at length in the common-dev@ mailing list, under topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment
[ https://issues.apache.org/jira/browse/HADOOP-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502221#comment-13502221 ] Matt Foley commented on HADOOP-9082: This discussion started in HADOOP-8924, where it was proposed to replace the build-time utility saveVersion.sh with a python script. This would require Python as a build-time dependency. Here's the background: Those of us involved in the branch-1-win port of Hadoop to Windows without use of Cygwin, have faced the issue of frequent use of shell scripts throughout the system, both in build time (eg, the utility saveVersion.sh), and run time (config files like hadoop-env.sh and the start/stop scripts in bin/* ). Similar usages exist throughout the Hadoop stack, in all projects. The vast majority of these shell scripts do not do anything platform specific; they can be expressed in a posix-conforming way. Therefore, it seems to us that it makes sense to start using a cross-platform scripting language, such as python, in place of shell for these purposes. For those rare occasions where platform-specific functionality really is needed, python also supports quite a lot of platform-specific functionality on both Linux and Windows; but where that is inadequate, one could still conditionally invoke a platform-specific module written in shell (for Linux/*nix) or powershell or bat (for Windows). The primary motive for moving to a cross-platform scripting language is maintainability. The alternative would be to maintain two complete suites of scripts, one for Linux and one for Windows (and perhaps others in the future). We want to avoid the need to update dual modules in two different languages when functionality changes, especially given that many Linux developers are not familiar with powershell or bat, and many Windows developers are not familiar with shell or bash. Select and document a platform-independent scripting language for use in Hadoop environment --- Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082 Project: Hadoop Common Issue Type: Bug Reporter: Matt Foley This issue is going to be discussed at length in the common-dev@ mailing list, under topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment
[ https://issues.apache.org/jira/browse/HADOOP-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502225#comment-13502225 ] Matt Foley commented on HADOOP-9082: Why python: * There are already a few instances of python usage in Hadoop, such as the utility (currently broken) relnotes.py, and massive usage of python in the examples/ and contrib/ directories. * Python is also used in Bigtop build-time. * The Python language is available for free on essentially all platforms, under an [Apache-compatible license|http://www.apache.org/legal/resolved.html]. * It is supported in Eclipse and similar IDEs. * Most importantly, it is widely accepted as a reasonably good OO scripting language, and it is easily learned by anyone who already knows shell or perl, or other common scripting languages. * On the [Tiobe index of programming language popularity|http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html], which seeks to measure the relative number of software engineers who know and use each language, Python far exceeds Perl and Ruby. The only more well-known scripting languages are PHP and Visual Basic, neither of which seems a prime candidate for this use. Select and document a platform-independent scripting language for use in Hadoop environment --- Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082 Project: Hadoop Common Issue Type: Bug Reporter: Matt Foley This issue is going to be discussed at length in the common-dev@ mailing list, under topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502231#comment-13502231 ] Matt Foley commented on HADOOP-9073: Please hold off on committing this until the discussion of HADOOP-9082 and the common-dev@ discussion topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack reaches a conclusion. Thank you. Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0 Reporter: Radim Kolar Attachments: version-py2.txt, version-py3.txt, version-py.txt Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment
[ https://issues.apache.org/jira/browse/HADOOP-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502233#comment-13502233 ] Radim Kolar commented on HADOOP-9082: - Groovy would be best. It requires no user installation. You can run groovy scripts directly from maven via groovy maven plugin and at runtime you need just to add groovy-all jar to classpath. Select and document a platform-independent scripting language for use in Hadoop environment --- Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082 Project: Hadoop Common Issue Type: Bug Reporter: Matt Foley This issue is going to be discussed at length in the common-dev@ mailing list, under topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9080) Update bylaws to specify PMC chair voting
[ https://issues.apache.org/jira/browse/HADOOP-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502239#comment-13502239 ] Tsz Wo (Nicholas), SZE commented on HADOOP-9080: +1 change looks good. Update bylaws to specify PMC chair voting - Key: HADOOP-9080 URL: https://issues.apache.org/jira/browse/HADOOP-9080 Project: Hadoop Common Issue Type: Task Reporter: Eli Collins Assignee: Eli Collins Attachments: hadoop-9080.txt The PMC voted to recommend a new chair using STV, let's update the bylaws accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[
https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502242#comment-13502242
]
Hadoop QA commented on HADOOP-9076:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554546/pstree-update4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1791//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1791//console
This message is automatically generated.
Fix QA bot in YARN project
--
Key: HADOOP-9076
URL: https://issues.apache.org/jira/browse/HADOOP-9076
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Radim Kolar
Priority: Trivial
Attachments: pstree-update4.txt
purpose of this ticket is to run automatic tests on patches to YARN project
which have not working QA bot. There seems to be zero interest in fixing QA
bot to make it work for YARN subproject.
If you need to get your patch for YARN tested, just subscribe and attach it
to this ticket.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
[ https://issues.apache.org/jira/browse/HADOOP-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins reassigned HADOOP-9079: --- Assignee: Jimmy Xiang LocalDirAllocator throws ArithmeticException Key: HADOOP-9079 URL: https://issues.apache.org/jira/browse/HADOOP-9079 Project: Hadoop Common Issue Type: Bug Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-9079.patch 2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671] nodemanager.NMAuditLogger(150): USER=UnknownUserIP= OPERATION=Stop Container RequestTARGET=ContainerManagerImpl RESULT=FAILURE DESCRIPTION=Trying to stop unknown container! APPID=application_1353391620476_0001 CONTAINERID=container_1353391620476_0001_01_10 java.lang.ArithmeticException: / by zero at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment
[ https://issues.apache.org/jira/browse/HADOOP-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502247#comment-13502247 ] Doug Cutting commented on HADOOP-9082: -- So on Windows this replaces the dependency on Cygwin with one on Python. Why is Cygwin unacceptable? Select and document a platform-independent scripting language for use in Hadoop environment --- Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082 Project: Hadoop Common Issue Type: Bug Reporter: Matt Foley This issue is going to be discussed at length in the common-dev@ mailing list, under topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated HADOOP-9064:
-
Attachment: hadoop-9064.patch
Posting the patch that augments the API as in the description.
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated HADOOP-9064:
-
Status: Patch Available (was: Open)
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated HADOOP-9064:
-
Attachment: hadoop-9064.patch
Changed the logging in removeRenewAction()
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch, hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8931) Add Java version to startup message
[ https://issues.apache.org/jira/browse/HADOOP-8931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated HADOOP-8931: -- Fix Version/s: 0.23.6 I pulled this into branch-0.23 too. Add Java version to startup message Key: HADOOP-8931 URL: https://issues.apache.org/jira/browse/HADOOP-8931 Project: Hadoop Common Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0-alpha Reporter: Eli Collins Assignee: Eli Collins Priority: Trivial Fix For: 1.2.0, 2.0.3-alpha, 0.23.6 Attachments: hadoop-8931-b1.txt, hadoop-8931.txt I often look at logs and have to track down the java version they were run with, it would be useful if we logged this as part of the startup message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502322#comment-13502322
]
Alejandro Abdelnur commented on HADOOP-9064:
I don't see a reason for removeRenewAction() to have param not to cancel, the
current usage of this is always t cancel. I'd just cancel and if we ever need a
conditional cancel, then we introduce the boolean param.
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch, hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment
[ https://issues.apache.org/jira/browse/HADOOP-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9082: Attachment: pstree-update5.txt Select and document a platform-independent scripting language for use in Hadoop environment --- Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082 Project: Hadoop Common Issue Type: Bug Reporter: Matt Foley This issue is going to be discussed at length in the common-dev@ mailing list, under topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment
[ https://issues.apache.org/jira/browse/HADOOP-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9082: Attachment: (was: pstree-update5.txt) Select and document a platform-independent scripting language for use in Hadoop environment --- Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082 Project: Hadoop Common Issue Type: Bug Reporter: Matt Foley This issue is going to be discussed at length in the common-dev@ mailing list, under topic [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: pstree-update5.txt Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update4.txt, pstree-update5.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502329#comment-13502329 ] Radim Kolar commented on HADOOP-9076: - There is zero QA bot activity in YARN project. It is still broken. Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update4.txt, pstree-update5.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8934) Shell command ls should include sort options
[ https://issues.apache.org/jira/browse/HADOOP-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-8934: --- Status: In Progress (was: Patch Available) actioning review comments Shell command ls should include sort options Key: HADOOP-8934 URL: https://issues.apache.org/jira/browse/HADOOP-8934 Project: Hadoop Common Issue Type: Improvement Components: fs Reporter: Jonathan Allen Assignee: Jonathan Allen Priority: Minor Attachments: HADOOP-8934.patch, HADOOP-8934.patch The shell command ls should include options to sort the output similar to the unix ls command. The following options seem appropriate: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : use access time rather than modification time for sort and display -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-9070:
Attachment: HADOOP-9070.patch
bump rpc version
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch, HADOOP-9070.patch, HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7713) dfs -count -q should label output column
[
https://issues.apache.org/jira/browse/HADOOP-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Allen updated HADOOP-7713:
---
Status: In Progress (was: Patch Available)
dfs -count -q should label output column
Key: HADOOP-7713
URL: https://issues.apache.org/jira/browse/HADOOP-7713
Project: Hadoop Common
Issue Type: Improvement
Reporter: Nigel Daley
Assignee: Jonathan Allen
Priority: Trivial
Labels: newbie
Attachments: HADOOP-7713.patch, HADOOP-7713.patch, HADOOP-7713.patch,
HADOOP-7713.patch, HADOOP-7713.patch, HADOOP-7713.patch
These commands should label the output columns:
{code}
hadoop dfs -count dir...dir
hadoop dfs -count -q dir...dir
{code}
Current output of the 2nd command above:
{code}
% hadoop dfs -count -q /user/foo /tmp
none inf 9569 9493 6372553322
hdfs://nn1.bar.com/user/foo
none inf 101 2689 209349812906
hdfs://nn1.bar.com/tmp
{code}
It is not obvious what these columns mean.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502361#comment-13502361
]
Hadoop QA commented on HADOOP-9064:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554561/hadoop-9064.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1792//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1792//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1792//console
This message is automatically generated.
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch, hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated HADOOP-9064:
-
Attachment: hadoop-9064.patch
Thanks Alejandro. Here is an updated patch incorporating your comments.
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch, hadoop-9064.patch, hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8615) EOFException in DecompressorStream.java needs to be more verbose
[
https://issues.apache.org/jira/browse/HADOOP-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502385#comment-13502385
]
Andy Isaacson commented on HADOOP-8615:
---
The latest version of the patch looks great. +1.
EOFException in DecompressorStream.java needs to be more verbose
Key: HADOOP-8615
URL: https://issues.apache.org/jira/browse/HADOOP-8615
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 0.20.2
Reporter: Jeff Lord
Labels: patch
Attachments: HADOOP-8615.patch, HADOOP-8615-release-0.20.2.patch,
HADOOP-8615-ver2.patch, HADOOP-8615-ver3.patch
In ./src/core/org/apache/hadoop/io/compress/DecompressorStream.java
The following exception should at least pass back the file that it encounters
this error in relation to:
protected void getCompressedData() throws IOException {
checkStream();
int n = in.read(buffer, 0, buffer.length);
if (n == -1) {
throw new EOFException(Unexpected end of input stream);
}
This would help greatly to debug bad/corrupt files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9079) LocalDirAllocator throws ArithmeticException
[ https://issues.apache.org/jira/browse/HADOOP-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502386#comment-13502386 ] Jimmy Xiang commented on HADOOP-9079: - Yes, some local dir should be out of space to reproduce this issue. Is there a good way to simulate a disk out of space? LocalDirAllocator throws ArithmeticException Key: HADOOP-9079 URL: https://issues.apache.org/jira/browse/HADOOP-9079 Project: Hadoop Common Issue Type: Bug Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-9079.patch 2012-11-19 22:07:41,709 WARN [IPC Server handler 0 on 38671] nodemanager.NMAuditLogger(150): USER=UnknownUserIP= OPERATION=Stop Container RequestTARGET=ContainerManagerImpl RESULT=FAILURE DESCRIPTION=Trying to stop unknown container! APPID=application_1353391620476_0001 CONTAINERID=container_1353391620476_0001_01_10 java.lang.ArithmeticException: / by zero at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:368) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:263) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502396#comment-13502396
]
Hadoop QA commented on HADOOP-9064:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554566/hadoop-9064.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1793//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1793//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1793//console
This message is automatically generated.
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch, hadoop-9064.patch, hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[
https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502404#comment-13502404
]
Hadoop QA commented on HADOOP-9076:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554574/pstree-update5.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1794//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1794//console
This message is automatically generated.
Fix QA bot in YARN project
--
Key: HADOOP-9076
URL: https://issues.apache.org/jira/browse/HADOOP-9076
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Radim Kolar
Priority: Trivial
Attachments: pstree-update4.txt, pstree-update5.txt
purpose of this ticket is to run automatic tests on patches to YARN project
which have not working QA bot. There seems to be zero interest in fixing QA
bot to make it work for YARN subproject.
If you need to get your patch for YARN tested, just subscribe and attach it
to this ticket.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: pstree-update4.txt) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update6.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: pstree-update5.txt) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update6.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: pstree-update6.txt Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial Attachments: pstree-update6.txt purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8934) Shell command ls should include sort options
[ https://issues.apache.org/jira/browse/HADOOP-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-8934: --- Attachment: HADOOP-8934.patch review comments actioned: - sort order for -S corrected - now using PathData.compareTo rather than Path - changes to PathData ( Command) reverted (and a MockFileSystem used in unit tests instead) I didn't know about TestHDFSCLI but have now confirmed change doesn't break any of the existing tests. I'll add some additional tests later this week. Shell command ls should include sort options Key: HADOOP-8934 URL: https://issues.apache.org/jira/browse/HADOOP-8934 Project: Hadoop Common Issue Type: Improvement Components: fs Reporter: Jonathan Allen Assignee: Jonathan Allen Priority: Minor Attachments: HADOOP-8934.patch, HADOOP-8934.patch, HADOOP-8934.patch The shell command ls should include options to sort the output similar to the unix ls command. The following options seem appropriate: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : use access time rather than modification time for sort and display -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9064) Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction
[
https://issues.apache.org/jira/browse/HADOOP-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502436#comment-13502436
]
Hadoop QA commented on HADOOP-9064:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554587/hadoop-9064.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1795//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1795//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1795//console
This message is automatically generated.
Augment DelegationTokenRenewer API to cancel the tokens on calls to
removeRenewAction
-
Key: HADOOP-9064
URL: https://issues.apache.org/jira/browse/HADOOP-9064
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: hadoop-9064.patch, hadoop-9064.patch, hadoop-9064.patch
Post HADOOP-9049, FileSystems register with DelegationTokenRenewer (a
singleton), to renew tokens.
To avoid a bunch of defunct tokens clog the NN, we should augment the API to
{{#removeRenewAction(boolean cancel)}} and cancel the token appropriately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502474#comment-13502474
]
Hadoop QA commented on HADOOP-9070:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554577/HADOOP-9070.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1796//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1796//console
This message is automatically generated.
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch, HADOOP-9070.patch, HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[
https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502515#comment-13502515
]
Hadoop QA commented on HADOOP-9076:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554601/pstree-update6.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1797//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1797//console
This message is automatically generated.
Fix QA bot in YARN project
--
Key: HADOOP-9076
URL: https://issues.apache.org/jira/browse/HADOOP-9076
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Radim Kolar
Priority: Trivial
Attachments: pstree-update6.txt
purpose of this ticket is to run automatic tests on patches to YARN project
which have not working QA bot. There seems to be zero interest in fixing QA
bot to make it work for YARN subproject.
If you need to get your patch for YARN tested, just subscribe and attach it
to this ticket.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9083) Port HADOOP-9020 Add a SASL PLAIN server to branch 1
Yu Gao created HADOOP-9083: -- Summary: Port HADOOP-9020 Add a SASL PLAIN server to branch 1 Key: HADOOP-9083 URL: https://issues.apache.org/jira/browse/HADOOP-9083 Project: Hadoop Common Issue Type: Task Components: ipc, security Affects Versions: 1.0.3 Reporter: Yu Gao Assignee: Yu Gao It would be good if the patch of HADOOP-9020 for adding SASL PLAIN server implementation could be ported to branch 1 as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9083) Port HADOOP-9020 Add a SASL PLAIN server to branch 1
[ https://issues.apache.org/jira/browse/HADOOP-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Gao updated HADOOP-9083: --- Attachment: HADOOP-9020-branch-1.patch Port HADOOP-9020 Add a SASL PLAIN server to branch 1 Key: HADOOP-9083 URL: https://issues.apache.org/jira/browse/HADOOP-9083 Project: Hadoop Common Issue Type: Task Components: ipc, security Affects Versions: 1.0.3 Reporter: Yu Gao Assignee: Yu Gao Attachments: HADOOP-9020-branch-1.patch It would be good if the patch of HADOOP-9020 for adding SASL PLAIN server implementation could be ported to branch 1 as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9083) Port HADOOP-9020 Add a SASL PLAIN server to branch 1
[ https://issues.apache.org/jira/browse/HADOOP-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Gao updated HADOOP-9083: --- Attachment: test-TestSaslRPC.result test-patch.result Port HADOOP-9020 Add a SASL PLAIN server to branch 1 Key: HADOOP-9083 URL: https://issues.apache.org/jira/browse/HADOOP-9083 Project: Hadoop Common Issue Type: Task Components: ipc, security Affects Versions: 1.0.3 Reporter: Yu Gao Assignee: Yu Gao Attachments: HADOOP-9020-branch-1.patch, test-patch.result, test-TestSaslRPC.result It would be good if the patch of HADOOP-9020 for adding SASL PLAIN server implementation could be ported to branch 1 as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9083) Port HADOOP-9020 Add a SASL PLAIN server to branch 1
[ https://issues.apache.org/jira/browse/HADOOP-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502540#comment-13502540 ] Yu Gao commented on HADOOP-9083: Attached ant test and test-patch results against branch 1. Also put the overall result here: ant test-patch: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.1) warnings. ant test -Dtestcase=TestSaslRPC Testsuite: org.apache.hadoop.ipc.TestSaslRPC Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 1.392 sec Port HADOOP-9020 Add a SASL PLAIN server to branch 1 Key: HADOOP-9083 URL: https://issues.apache.org/jira/browse/HADOOP-9083 Project: Hadoop Common Issue Type: Task Components: ipc, security Affects Versions: 1.0.3 Reporter: Yu Gao Assignee: Yu Gao Attachments: HADOOP-9020-branch-1.patch, test-patch.result, test-TestSaslRPC.result It would be good if the patch of HADOOP-9020 for adding SASL PLAIN server implementation could be ported to branch 1 as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9083) Port HADOOP-9020 Add a SASL PLAIN server to branch 1
[ https://issues.apache.org/jira/browse/HADOOP-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Gao updated HADOOP-9083: --- Attachment: test-patch.result Port HADOOP-9020 Add a SASL PLAIN server to branch 1 Key: HADOOP-9083 URL: https://issues.apache.org/jira/browse/HADOOP-9083 Project: Hadoop Common Issue Type: Task Components: ipc, security Affects Versions: 1.0.3 Reporter: Yu Gao Assignee: Yu Gao Attachments: HADOOP-9020-branch-1.patch, test-patch.result, test-TestSaslRPC.result It would be good if the patch of HADOOP-9020 for adding SASL PLAIN server implementation could be ported to branch 1 as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9083) Port HADOOP-9020 Add a SASL PLAIN server to branch 1
[ https://issues.apache.org/jira/browse/HADOOP-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Gao updated HADOOP-9083: --- Attachment: (was: test-patch.result) Port HADOOP-9020 Add a SASL PLAIN server to branch 1 Key: HADOOP-9083 URL: https://issues.apache.org/jira/browse/HADOOP-9083 Project: Hadoop Common Issue Type: Task Components: ipc, security Affects Versions: 1.0.3 Reporter: Yu Gao Assignee: Yu Gao Attachments: HADOOP-9020-branch-1.patch, test-patch.result, test-TestSaslRPC.result It would be good if the patch of HADOOP-9020 for adding SASL PLAIN server implementation could be ported to branch 1 as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: pstree-update6.txt) Fix QA bot in YARN project -- Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Radim Kolar Priority: Trivial purpose of this ticket is to run automatic tests on patches to YARN project which have not working QA bot. There seems to be zero interest in fixing QA bot to make it work for YARN subproject. If you need to get your patch for YARN tested, just subscribe and attach it to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9084) TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem)
giovanni delussu created HADOOP-9084: Summary: TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem) Key: HADOOP-9084 URL: https://issues.apache.org/jira/browse/HADOOP-9084 Project: Hadoop Common Issue Type: Bug Components: filecache, fs, native Affects Versions: 1.0.4, 1.0.3 Reporter: giovanni delussu Priority: Critical Fix For: 1.0.4, 1.0.3 When running a job who uses TotalOrderPartitioner (like TeraSort or BulkImport of HBase) on hadoop running on top of gpfs (instead of hdfs) the program fails to find the file _partition.lst because is looking for it in the wrong directory. The confusion is between local fs meaning not hdfs and local meaning distributed fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9084) TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem)
[ https://issues.apache.org/jira/browse/HADOOP-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] giovanni delussu updated HADOOP-9084: - Assignee: giovanni delussu TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem) Key: HADOOP-9084 URL: https://issues.apache.org/jira/browse/HADOOP-9084 Project: Hadoop Common Issue Type: Bug Components: filecache, fs, native Affects Versions: 1.0.3, 1.0.4 Reporter: giovanni delussu Assignee: giovanni delussu Priority: Critical Fix For: 1.0.3, 1.0.4 When running a job who uses TotalOrderPartitioner (like TeraSort or BulkImport of HBase) on hadoop running on top of gpfs (instead of hdfs) the program fails to find the file _partition.lst because is looking for it in the wrong directory. The confusion is between local fs meaning not hdfs and local meaning distributed fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9084) TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem)
[ https://issues.apache.org/jira/browse/HADOOP-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] giovanni delussu updated HADOOP-9084: - Status: Patch Available (was: Open) *This patch works by keeping the fs and the jobconf working directories different. In hdfs we don't need it because the local fs is different from the default fs. *potential problems of this patch: if a user creates a file ( with relative path) in the mapper/reducer phase this file will be created in the attempt directory and then deleted on cleanup. Without the patch the file would be created in the current working directory of the filesystem, i.e. the directory from which the user launched hadoop. TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem) Key: HADOOP-9084 URL: https://issues.apache.org/jira/browse/HADOOP-9084 Project: Hadoop Common Issue Type: Bug Components: filecache, fs, native Affects Versions: 1.0.4, 1.0.3 Reporter: giovanni delussu Assignee: giovanni delussu Priority: Critical Fix For: 1.0.4, 1.0.3 When running a job who uses TotalOrderPartitioner (like TeraSort or BulkImport of HBase) on hadoop running on top of gpfs (instead of hdfs) the program fails to find the file _partition.lst because is looking for it in the wrong directory. The confusion is between local fs meaning not hdfs and local meaning distributed fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9084) TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem)
[ https://issues.apache.org/jira/browse/HADOOP-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] giovanni delussu updated HADOOP-9084: - Attachment: PATCH-HADOOP-9084-origin-branch-1.0.patch TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem) Key: HADOOP-9084 URL: https://issues.apache.org/jira/browse/HADOOP-9084 Project: Hadoop Common Issue Type: Bug Components: filecache, fs, native Affects Versions: 1.0.3, 1.0.4 Reporter: giovanni delussu Assignee: giovanni delussu Priority: Critical Fix For: 1.0.3, 1.0.4 Attachments: PATCH-HADOOP-9084-origin-branch-1.0.patch When running a job who uses TotalOrderPartitioner (like TeraSort or BulkImport of HBase) on hadoop running on top of gpfs (instead of hdfs) the program fails to find the file _partition.lst because is looking for it in the wrong directory. The confusion is between local fs meaning not hdfs and local meaning distributed fs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
[ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502624#comment-13502624 ] Yanbo Liang commented on HADOOP-9041: - As my tracing the process of running test case in maven+junit environment, I have found that in this environment the ServiceLoader will directly load service providers from the file target/class/META-INF/services/org.apache.hadoop.fs.FileSystem rather than load from the jar package. Because much resource would be loaded from this directory target/class/, so the classloader will reuse this URL and then reuse the URL handler for the scheme of file. ServiceLoader will call classloader to load resource from the file in this directory, then the loading will not use the the handler we just register. Due to not use the handler we just register in FsUrlStreamHandlerFactory, the reusing handler will work well and not fall into the infinite loop. So this bug may be difficult to reproduce in junit environment. But in regular development, it will be dangerous and hidden deep. Until here I think my patch can resolve this problem and looking forward any comments. FileSystem initialization can go into infinite loop --- Key: HADOOP-9041 URL: https://issues.apache.org/jira/browse/HADOOP-9041 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Radim Kolar Assignee: Yanbo Liang Priority: Critical Attachments: fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, HADOOP-9041.patch More information is there: https://jira.springsource.org/browse/SHDP-111 Referenced source code from example is: https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java from isolating that cause it looks like if you register: org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling FileSystem.loadFileSystems() then it goes into infinite loop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
