[jira] [Updated] (MAPREDUCE-3752) Headroom should be capped by queue max-cap
[ https://issues.apache.org/jira/browse/MAPREDUCE-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3752: - Attachment: MAPREDUCE-3752.patch Ready for review/commit. Headroom should be capped by queue max-cap -- Key: MAPREDUCE-3752 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3752 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3752.patch Headroom should be capped by queue max-cap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3752) Headroom should be capped by queue max-cap
[ https://issues.apache.org/jira/browse/MAPREDUCE-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3752: - Status: Patch Available (was: Open) Headroom should be capped by queue max-cap -- Key: MAPREDUCE-3752 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3752 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3752.patch Headroom should be capped by queue max-cap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3752) Headroom should be capped by queue max-cap
[ https://issues.apache.org/jira/browse/MAPREDUCE-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195471#comment-13195471 ] Hadoop QA commented on MAPREDUCE-3752: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512298/MAPREDUCE-3752.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestJobClientGetJob org.apache.hadoop.mapred.TestMRWithDistributedCache org.apache.hadoop.mapred.TestLocalModeWithNewApis +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1702//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1702//console This message is automatically generated. Headroom should be capped by queue max-cap -- Key: MAPREDUCE-3752 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3752 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3752.patch Headroom should be capped by queue max-cap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3752) Headroom should be capped by queue max-cap
[ https://issues.apache.org/jira/browse/MAPREDUCE-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195483#comment-13195483 ] Hadoop QA commented on MAPREDUCE-3752: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512298/MAPREDUCE-3752.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1703//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1703//console This message is automatically generated. Headroom should be capped by queue max-cap -- Key: MAPREDUCE-3752 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3752 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3752.patch Headroom should be capped by queue max-cap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3740) Mapreduce Trunk compilation fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195505#comment-13195505
]
Hudson commented on MAPREDUCE-3740:
---
Integrated in Hadoop-Hdfs-trunk #939 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/939/])
MAPREDUCE-3740. Fixed broken mapreduce compilation after the patch for
HADOOP-7965. Contributed by Devaraj K.
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236901
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java
Mapreduce Trunk compilation fails
-
Key: MAPREDUCE-3740
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3740
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Fix For: 0.24.0
Attachments: MAPREDUCE-3740.patch
{code:xml}
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/home/hadoop/hadoop-trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java:[61,7]
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine is not abstract and does
not override abstract method
getProtocolMetaInfoProxy(org.apache.hadoop.ipc.Client.ConnectionId,org.apache.hadoop.conf.Configuration,javax.net.SocketFactory)
in org.apache.hadoop.ipc.RpcEngine
[INFO] 1 error
[INFO] -
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3742) yarn logs command fails with ClassNotFoundException
[ https://issues.apache.org/jira/browse/MAPREDUCE-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195508#comment-13195508 ] Hudson commented on MAPREDUCE-3742: --- Integrated in Hadoop-Hdfs-trunk #939 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/939/]) MAPREDUCE-3742. yarn logs command fails with ClassNotFoundException. (Jason Lowe via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236816 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn yarn logs command fails with ClassNotFoundException - Key: MAPREDUCE-3742 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3742 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jason Lowe Assignee: Jason Lowe Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3742.patch Executing yarn logs at a shell prompt fails with this error: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper. Program will exit. Appears to have been caused by the code reorg in MAPREDUCE-3297. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3732) CS should only use 'activeUsers with pending requests' for computing user-limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195511#comment-13195511 ] Hudson commented on MAPREDUCE-3732: --- Integrated in Hadoop-Hdfs-trunk #939 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/939/]) MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending requests for computing user-limits. Contributed by Arun C Murthy. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236953 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ActiveUsersManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java CS should only use 'activeUsers with pending requests' for computing user-limits Key: MAPREDUCE-3732 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3732 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, resourcemanager, scheduler Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3732.patch, MAPREDUCE-3732.patch, MAPREDUCE-3732.patch CS should only use 'activeUsers with pending requests' for computing user-limits, similar to what is done in hadoop-1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3742) yarn logs command fails with ClassNotFoundException
[ https://issues.apache.org/jira/browse/MAPREDUCE-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195519#comment-13195519 ] Hudson commented on MAPREDUCE-3742: --- Integrated in Hadoop-Mapreduce-0.23-Build #174 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/174/]) MAPREDUCE-3742. yarn logs command fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1236816 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236818 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn yarn logs command fails with ClassNotFoundException - Key: MAPREDUCE-3742 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3742 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jason Lowe Assignee: Jason Lowe Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3742.patch Executing yarn logs at a shell prompt fails with this error: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper. Program will exit. Appears to have been caused by the code reorg in MAPREDUCE-3297. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3732) CS should only use 'activeUsers with pending requests' for computing user-limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195522#comment-13195522 ] Hudson commented on MAPREDUCE-3732: --- Integrated in Hadoop-Mapreduce-0.23-Build #174 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/174/]) MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending requests for computing user-limits. Contributed by Arun C Murthy. svn merge --ignore-ancestry -c 1236953 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236954 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ActiveUsersManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java CS should only use 'activeUsers with pending requests' for computing user-limits Key: MAPREDUCE-3732 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3732 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, resourcemanager, scheduler Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3732.patch, MAPREDUCE-3732.patch, MAPREDUCE-3732.patch CS should only use 'activeUsers with pending requests' for computing user-limits, similar to what is done in hadoop-1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3742) yarn logs command fails with ClassNotFoundException
[ https://issues.apache.org/jira/browse/MAPREDUCE-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195528#comment-13195528 ] Hudson commented on MAPREDUCE-3742: --- Integrated in Hadoop-Hdfs-0.23-Build #152 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/152/]) MAPREDUCE-3742. yarn logs command fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1236816 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236818 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn yarn logs command fails with ClassNotFoundException - Key: MAPREDUCE-3742 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3742 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jason Lowe Assignee: Jason Lowe Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3742.patch Executing yarn logs at a shell prompt fails with this error: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper. Program will exit. Appears to have been caused by the code reorg in MAPREDUCE-3297. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3732) CS should only use 'activeUsers with pending requests' for computing user-limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195531#comment-13195531 ] Hudson commented on MAPREDUCE-3732: --- Integrated in Hadoop-Hdfs-0.23-Build #152 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/152/]) MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending requests for computing user-limits. Contributed by Arun C Murthy. svn merge --ignore-ancestry -c 1236953 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236954 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ActiveUsersManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java CS should only use 'activeUsers with pending requests' for computing user-limits Key: MAPREDUCE-3732 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3732 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, resourcemanager, scheduler Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3732.patch, MAPREDUCE-3732.patch, MAPREDUCE-3732.patch CS should only use 'activeUsers with pending requests' for computing user-limits, similar to what is done in hadoop-1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2539) NPE when calling JobClient.getMapTaskReports for retired job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195537#comment-13195537 ] Hudson commented on MAPREDUCE-2539: --- Integrated in Hadoop-Mapreduce-22-branch #98 (See [https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/98/]) MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient. Contributed by Robert Evans. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236961 Files : * /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt * /hadoop/common/branches/branch-0.22/mapreduce/src/java/org/apache/hadoop/mapred/JobClient.java * /hadoop/common/branches/branch-0.22/mapreduce/src/test/mapred/org/apache/hadoop/mapred/JobClientUnitTest.java NPE when calling JobClient.getMapTaskReports for retired job Key: MAPREDUCE-2539 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2539 Project: Hadoop Map/Reduce Issue Type: Bug Components: client Affects Versions: 0.22.0 Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Attachments: MR-2539-trunk-v1.patch, MR-2539-yarn-v1.patch When calling JobClient.getMapTaskReports for a retired job this results in a NPE. In the 0.20.* version an empty TaskReport array was returned instead. Caused by: java.lang.NullPointerException at org.apache.hadoop.mapred.JobClient.getMapTaskReports(JobClient.java:588) at org.apache.pig.tools.pigstats.JobStats.addMapReduceStatistics(JobStats.java:388) .. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Attachment: MAPREDUCE-3034.patch I have updated the first level patch for review. Please review and give your comments. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Attachments: MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Reopened) NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Attachments: MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3746) Nodemanagers are not automatically shut down after decommissioning
[ https://issues.apache.org/jira/browse/MAPREDUCE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K reassigned MAPREDUCE-3746: Assignee: Devaraj K Nodemanagers are not automatically shut down after decommissioning -- Key: MAPREDUCE-3746 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3746 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Devaraj K Priority: Critical Fix For: 0.23.1 Nodemanagers are not automatically shutdown after decommissioning. MAPREDUCE-2775 does not seem to fix the issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3740) Mapreduce Trunk compilation fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195548#comment-13195548
]
Hudson commented on MAPREDUCE-3740:
---
Integrated in Hadoop-Mapreduce-trunk #972 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/972/])
MAPREDUCE-3740. Fixed broken mapreduce compilation after the patch for
HADOOP-7965. Contributed by Devaraj K.
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236901
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java
Mapreduce Trunk compilation fails
-
Key: MAPREDUCE-3740
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3740
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Fix For: 0.24.0
Attachments: MAPREDUCE-3740.patch
{code:xml}
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/home/hadoop/hadoop-trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java:[61,7]
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine is not abstract and does
not override abstract method
getProtocolMetaInfoProxy(org.apache.hadoop.ipc.Client.ConnectionId,org.apache.hadoop.conf.Configuration,javax.net.SocketFactory)
in org.apache.hadoop.ipc.RpcEngine
[INFO] 1 error
[INFO] -
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3732) CS should only use 'activeUsers with pending requests' for computing user-limits
[ https://issues.apache.org/jira/browse/MAPREDUCE-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195554#comment-13195554 ] Hudson commented on MAPREDUCE-3732: --- Integrated in Hadoop-Mapreduce-trunk #972 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/972/]) MAPREDUCE-3732. Modified CapacityScheduler to use only users with pending requests for computing user-limits. Contributed by Arun C Murthy. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236953 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ActiveUsersManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java CS should only use 'activeUsers with pending requests' for computing user-limits Key: MAPREDUCE-3732 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3732 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, resourcemanager, scheduler Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3732.patch, MAPREDUCE-3732.patch, MAPREDUCE-3732.patch CS should only use 'activeUsers with pending requests' for computing user-limits, similar to what is done in hadoop-1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3742) yarn logs command fails with ClassNotFoundException
[ https://issues.apache.org/jira/browse/MAPREDUCE-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195551#comment-13195551 ] Hudson commented on MAPREDUCE-3742: --- Integrated in Hadoop-Mapreduce-trunk #972 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/972/]) MAPREDUCE-3742. yarn logs command fails with ClassNotFoundException. (Jason Lowe via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1236816 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn yarn logs command fails with ClassNotFoundException - Key: MAPREDUCE-3742 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3742 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jason Lowe Assignee: Jason Lowe Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3742.patch Executing yarn logs at a shell prompt fails with this error: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogDumper. Program will exit. Appears to have been caused by the code reorg in MAPREDUCE-3297. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195558#comment-13195558 ] Hadoop QA commented on MAPREDUCE-3034: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512311/MAPREDUCE-3034.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestJobClientGetJob org.apache.hadoop.mapred.TestMRWithDistributedCache org.apache.hadoop.mapred.TestLocalModeWithNewApis +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1704//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1704//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1704//console This message is automatically generated. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Attachments: MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2903) Map Tasks graph is throwing XML Parse error when Job is executed with 0 maps
[
https://issues.apache.org/jira/browse/MAPREDUCE-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-2903:
-
Fix Version/s: 1.1.0
Map Tasks graph is throwing XML Parse error when Job is executed with 0 maps
Key: MAPREDUCE-2903
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2903
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.205.0
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 1.1.0
Attachments: MAPREDUCE-2903.patch
{code:xml}
XML Parsing Error: no element found
Location:
http://10.18.52.170:50030/taskgraph?type=mapjobid=job_201108291536_0001
Line Number 1, Column 1:
^
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2721) TaskTracker is calling the close () method twice unnecessarily
[ https://issues.apache.org/jira/browse/MAPREDUCE-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K resolved MAPREDUCE-2721. -- Resolution: Won't Fix It is no more problem in 0.23 and trunk. TaskTracker is calling the close () method twice unnecessarily --- Key: MAPREDUCE-2721 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2721 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.23.0 Reporter: Devaraj K Assignee: Devaraj K Attachments: MAPREDUCE-2721.patch When there is a version mismatch between the JobTracker and TaskTracker, TaskTracker is calling the close () method twice. Once in the finally block of run () method and also in the shutdown () method call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2513) Improvements in Job Tracker UI for monitoring and managing the map reduce jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K resolved MAPREDUCE-2513. -- Resolution: Not A Problem It is not a problem anymore. Improvements in Job Tracker UI for monitoring and managing the map reduce jobs -- Key: MAPREDUCE-2513 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2513 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Devaraj K Assignee: Devaraj K It will be helpful to the user/administrator if we provide following features in the Job Tracker UI 1. User wants to get the list of jobs submitted with given state 2. User wants to kill a scheduled/running job through UI 3. User wants to change the priority of a job 4. User wants to get the scheduling information of jobs 5. User wants to delete the logs of Jobs and tasks 6. Only authorized users to be able to perform the above operations through task management UI 7. Pagination support for the jobs listing -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2525) State of the checkboxes are not matching with Select All / Deselect All button, after refreshing jobtracker.jsp
[ https://issues.apache.org/jira/browse/MAPREDUCE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K resolved MAPREDUCE-2525. -- Resolution: Won't Fix It is no more issue in 0.23 and trunk. State of the checkboxes are not matching with Select All / Deselect All button, after refreshing jobtracker.jsp --- Key: MAPREDUCE-2525 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2525 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.23.0 Environment: *Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3* Reporter: Devaraj K Assignee: Devaraj K Priority: Minor Fix For: 0.24.0 Attachments: MAPREDUCE-2525.patch, MR-UI.jpg These are the steps to reproduce, # Select all the running jobs. # Refresh the jobtracker.jsp page. # After refreshing, *Deselect All* button becomes *Select All*, however all the selected check boxes remain in the select state only. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2346) JobClient's isSuccessful and isComplete API's can throw NPE in some cases
[
https://issues.apache.org/jira/browse/MAPREDUCE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-2346:
-
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
It is not a problem in active versions.
JobClient's isSuccessful and isComplete API's can throw NPE in some cases
-
Key: MAPREDUCE-2346
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2346
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 0.20.1
Reporter: Bhallamudi Venkata Siva Kamesh
Assignee: Devaraj K
Priority: Minor
Attachments: MAPREDUCE-2346.patch
Description:
* Submit a job to the job tracker and let the job complete its execution
through one of the job client's submitJob APIs.
* Jobclient returns a handle to the job, in the form of a RunningJob object.
Client can use this object to check whether job is sucessful or whether job
is completed.
* Reduce the following property *mapred.jobtracker.retirejob.interval*.By
default this value is 1 day. I reduced it to 5 min.
* Set the property *mapred.job.tracker.persist.jobstatus.active* to
{color:blue}*false*{color}.
* Call either isComplete or isSuccessful APIs, after
*mapred.jobtracker.retirejob.interval* time period, previously mentioned APIs
throw NPE.
Below I am attaching stack trace
{code:xml}
java.lang.NullPointerException
at
org.apache.hadoop.mapred.JobClient$NetworkedJob.isSuccessful(JobClient.java:330)
at
com.huawei.isap.hdp.mapreduce.test.TestJobClient.testjobClientForNULL(TestJobClient.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:164)
at junit.framework.TestCase.runBare(TestCase.java:130)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:120)
at junit.framework.TestSuite.runTest(TestSuite.java:230)
at junit.framework.TestSuite.run(TestSuite.java:225)
at
org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2310) If we stop Job Tracker, Task Tracker is also getting stopped.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K resolved MAPREDUCE-2310. -- Resolution: Won't Fix It is not a problem in active versions. If we stop Job Tracker, Task Tracker is also getting stopped. - Key: MAPREDUCE-2310 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2310 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 0.20.2 Reporter: Devaraj K Assignee: Devaraj K Priority: Minor Attachments: MAPREDUCE-2310.patch If we execute stop-jobtracker.sh for stopping Job Tracker, Task Tracker is also stopping. This is not applicable for the latest (trunk) code because stop-jobtracker.sh file is not coming. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-1541) JobHistory page should list job start time rather than job-tracker start time
[ https://issues.apache.org/jira/browse/MAPREDUCE-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K resolved MAPREDUCE-1541. -- Resolution: Not A Problem It is no more problem in active versions. JobHistory page should list job start time rather than job-tracker start time - Key: MAPREDUCE-1541 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1541 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Reporter: Arun C Murthy Assignee: Devaraj K Attachments: MAPREDUCE-1541.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3193) FileInputFormat doesn't read files recursively in the input path dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3193: - Fix Version/s: 1.1.0 FileInputFormat doesn't read files recursively in the input path dir Key: MAPREDUCE-3193 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3193 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.24.0 Reporter: Ramgopal N Assignee: Devaraj K Fix For: 0.24.0, 1.1.0 Attachments: MAPREDUCE-3193-1.patch, MAPREDUCE-3193-2.patch, MAPREDUCE-3193-2.patch, MAPREDUCE-3193.patch, MAPREDUCE-3193.security.patch java.io.FileNotFoundException is thrown,if input file is more than one folder level deep and the job is getting failed. Example:Input file is /r1/r2/input.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2159) Provide metrics for RaidNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2159: --- Status: Open (was: Patch Available) Canceling the patch as it is over a year old, and there are review comments to be addressed. Provide metrics for RaidNode Key: MAPREDUCE-2159 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2159 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali Attachments: MAPREDUCE-2159.patch It will be useful to have the following metrics for RAID: - files raided - files too new to be raided - files too small to be raided - number of blocks fixed using raid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1073) Progress reported for pipes tasks is incorrect.
[
https://issues.apache.org/jira/browse/MAPREDUCE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Joseph Evans updated MAPREDUCE-1073:
---
Status: Open (was: Patch Available)
Canceling the patch as the latest patch is over a year old, and will not apply
to trunk any more. Dick, if you are still interested in getting this patch in,
please up merge and repost it. I would be happy to review it, give you feed
back on your approach, and commit it for you. If you have abandoned the JIRA
please post a comment in the JIRA so I can close it for you.
Progress reported for pipes tasks is incorrect.
---
Key: MAPREDUCE-1073
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1073
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: pipes
Affects Versions: 0.20.1
Reporter: Sreekanth Ramakrishnan
Assignee: Dick King
Attachments: MAPREDUCE-1073--yhadoop20--2010-07-22--1530.patch,
MAPREDUCE-1073--yhadoop20--2010-07-22.patch, MAPREDUCE-1073_yhadoop20.patch,
mapreduce-1073--2010-03-31.patch, mapreduce-1073--2010-04-06.patch
Currently in pipes,
{{org.apache.hadoop.mapred.pipes.PipesMapRunner.run(RecordReaderK1, V1,
OutputCollectorK2, V2, Reporter)}} we do the following:
{code}
while (input.next(key, value)) {
downlink.mapItem(key, value);
if(skipping) {
downlink.flush();
}
}
{code}
This would result in consumption of all the records for current task and
taking task progress to 100% whereas the actual pipes application would be
trailing behind.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
