[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200734#comment-13200734 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Hdfs-trunk #947 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/947/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240598 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes --- Key: MAPREDUCE-3759 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3759-20120202.txt When there are a few blacklisted nodes in the cluster, bin/mapred job -list-active-trackers throws java.lang.ClassCastException: org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) job -status command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200733#comment-13200733 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Hdfs-trunk #947 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/947/]) MAPREDUCE-3795. job -status command line output is malformed. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240593 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java job -status command line output is malformed -- Key: MAPREDUCE-3795 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3795-20120202.txt Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200735#comment-13200735 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Hdfs-trunk #947 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/947/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240603 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java Change MiniYarnCluster to escape special chars in testname -- Key: MAPREDUCE-3775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MR-3775.1.patch When using MiniYarnCluster with the testname set to a nested classname, the $ within the class name creates issues with the container launch scripts as they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200737#comment-13200737
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Hdfs-trunk #947 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/947/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240587
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/trunk/hadoop-project/src/site/site.xml
can't build site in hadoop-yarn-server-common
-
Key: MAPREDUCE-3791
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: documentation, mrv2
Affects Versions: 0.23.1
Reporter: Roman Shaposhnik
Assignee: Mahadev konar
Fix For: 0.23.1
Attachments: MAPREDUCE-3791.patch
Here's how to reproduce:
{noformat}
$ mvn site site:stage -DskipTests -DskipTest -DskipITs
main:
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main SUCCESS [49.017s]
[INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
[INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
[INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
[INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
[INFO] Apache Hadoop Auth SUCCESS [5.215s]
[INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
[INFO] Apache Hadoop Common .. SUCCESS [4.111s]
[INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
[INFO] Apache Hadoop HDFS SUCCESS [5.373s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
[INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
[INFO] hadoop-yarn ... SUCCESS [5.250s]
[INFO] hadoop-yarn-api ... SUCCESS [4.579s]
[INFO] hadoop-yarn-common SUCCESS [4.268s]
[INFO] hadoop-yarn-server SUCCESS [4.408s]
[INFO] hadoop-yarn-server-common . FAILURE [0.035s]
[INFO] hadoop-yarn-server-nodemanager SKIPPED
[INFO] hadoop-yarn-server-web-proxy .. SKIPPED
[INFO] hadoop-yarn-server-resourcemanager SKIPPED
[INFO] hadoop-yarn-server-tests .. SKIPPED
[INFO] hadoop-mapreduce-client ... SKIPPED
[INFO] hadoop-mapreduce-client-core .. SKIPPED
[INFO] hadoop-yarn-applications .. SKIPPED
[INFO] hadoop-yarn-applications-distributedshell . SKIPPED
[INFO] hadoop-yarn-site .. SKIPPED
[INFO] hadoop-mapreduce-client-common SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
[INFO] hadoop-mapreduce-client-app ... SKIPPED
[INFO] hadoop-mapreduce-client-hs SKIPPED
[INFO] hadoop-mapreduce-client-jobclient . SKIPPED
[INFO] Apache Hadoop MapReduce Examples .. SKIPPED
[INFO] hadoop-mapreduce .. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming . SKIPPED
[INFO] Apache Hadoop Distributed Copy SKIPPED
[INFO] Apache Hadoop Archives SKIPPED
[INFO] Apache Hadoop Rumen ... SKIPPED
[INFO] Apache Hadoop Extras .. SKIPPED
[INFO] Apache Hadoop Tools Dist .. SKIPPED
[INFO] Apache Hadoop Tools ... SKIPPED
[INFO] Apache Hadoop Distribution SKIPPED
[INFO] Apache Hadoop Client .. SKIPPED
[INFO] Apache Hadoop Mini-Cluster SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 2:18.427s
[INFO] Finished at: Thu Feb 02 10:31:35 PST 2012
[INFO] Final Memory: 321M/1012M
[INFO]
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200745#comment-13200745 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Hdfs-0.23-Build #160 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/160/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) - Merging r1240634 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240635 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting Key: MAPREDUCE-3765 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch FifoScheduler uses default min 1 GB regardless of the configuration value set for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) job -status command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200741#comment-13200741 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Hdfs-0.23-Build #160 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/160/]) MAPREDUCE-3795. job -status command line output is malformed. (vinodkv via mahadev) - Merging r1240593 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240594 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java job -status command line output is malformed -- Key: MAPREDUCE-3795 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3795-20120202.txt Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200742#comment-13200742 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Hdfs-0.23-Build #160 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/160/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) - Merging r1240598 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240599 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes --- Key: MAPREDUCE-3759 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3759-20120202.txt When there are a few blacklisted nodes in the cluster, bin/mapred job -list-active-trackers throws java.lang.ClassCastException: org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200743#comment-13200743 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Hdfs-0.23-Build #160 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/160/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) - Merging r1240603 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240604 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java Change MiniYarnCluster to escape special chars in testname -- Key: MAPREDUCE-3775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MR-3775.1.patch When using MiniYarnCluster with the testname set to a nested classname, the $ within the class name creates issues with the container launch scripts as they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200744#comment-13200744
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Hdfs-0.23-Build #160 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/160/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev) -
Merging r1240587 from trunk
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240588
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site/site.xml
can't build site in hadoop-yarn-server-common
-
Key: MAPREDUCE-3791
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: documentation, mrv2
Affects Versions: 0.23.1
Reporter: Roman Shaposhnik
Assignee: Mahadev konar
Fix For: 0.23.1
Attachments: MAPREDUCE-3791.patch
Here's how to reproduce:
{noformat}
$ mvn site site:stage -DskipTests -DskipTest -DskipITs
main:
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main SUCCESS [49.017s]
[INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
[INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
[INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
[INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
[INFO] Apache Hadoop Auth SUCCESS [5.215s]
[INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
[INFO] Apache Hadoop Common .. SUCCESS [4.111s]
[INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
[INFO] Apache Hadoop HDFS SUCCESS [5.373s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
[INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
[INFO] hadoop-yarn ... SUCCESS [5.250s]
[INFO] hadoop-yarn-api ... SUCCESS [4.579s]
[INFO] hadoop-yarn-common SUCCESS [4.268s]
[INFO] hadoop-yarn-server SUCCESS [4.408s]
[INFO] hadoop-yarn-server-common . FAILURE [0.035s]
[INFO] hadoop-yarn-server-nodemanager SKIPPED
[INFO] hadoop-yarn-server-web-proxy .. SKIPPED
[INFO] hadoop-yarn-server-resourcemanager SKIPPED
[INFO] hadoop-yarn-server-tests .. SKIPPED
[INFO] hadoop-mapreduce-client ... SKIPPED
[INFO] hadoop-mapreduce-client-core .. SKIPPED
[INFO] hadoop-yarn-applications .. SKIPPED
[INFO] hadoop-yarn-applications-distributedshell . SKIPPED
[INFO] hadoop-yarn-site .. SKIPPED
[INFO] hadoop-mapreduce-client-common SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
[INFO] hadoop-mapreduce-client-app ... SKIPPED
[INFO] hadoop-mapreduce-client-hs SKIPPED
[INFO] hadoop-mapreduce-client-jobclient . SKIPPED
[INFO] Apache Hadoop MapReduce Examples .. SKIPPED
[INFO] hadoop-mapreduce .. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming . SKIPPED
[INFO] Apache Hadoop Distributed Copy SKIPPED
[INFO] Apache Hadoop Archives SKIPPED
[INFO] Apache Hadoop Rumen ... SKIPPED
[INFO] Apache Hadoop Extras .. SKIPPED
[INFO] Apache Hadoop Tools Dist .. SKIPPED
[INFO] Apache Hadoop Tools ... SKIPPED
[INFO] Apache Hadoop Distribution SKIPPED
[INFO] Apache Hadoop Client .. SKIPPED
[INFO] Apache Hadoop Mini-Cluster SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 2:18.427s
[INFO] Finished at: Thu Feb
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200746#comment-13200746
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Hdfs-0.23-Build #160 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/160/])
MAPREDUCE-3723. TestAMWebServicesJobs TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) - Merging r1240590
from trunk.
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240591
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
TestAMWebServicesJobs TestHSWebServicesJobs incorrectly asserting tests
-
Key: MAPREDUCE-3723
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, test, webapps
Affects Versions: 0.23.1, 0.24.0
Reporter: Bhallamudi Venkata Siva Kamesh
Assignee: Bhallamudi Venkata Siva Kamesh
Fix For: 0.23.1
Attachments: MAPREDUCE-3723.patch
While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
TestHSWebServicesJobs incorrectly asserting tests.
Moreover tests may fail if
{noformat}
index of counterGroups #counters in a particular counterGroup
{noformat}
{code:title=TestAMWebServicesJobs.java|borderStyle=solid}
for (int j = 0; j counters.length(); j++) {
JSONObject counter = counters.getJSONObject(i);
{code}
where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) job -status command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200756#comment-13200756 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Mapreduce-0.23-Build #182 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/182/]) MAPREDUCE-3795. job -status command line output is malformed. (vinodkv via mahadev) - Merging r1240593 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240594 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java job -status command line output is malformed -- Key: MAPREDUCE-3795 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3795-20120202.txt Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200757#comment-13200757 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Mapreduce-0.23-Build #182 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/182/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) - Merging r1240598 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240599 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes --- Key: MAPREDUCE-3759 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3759-20120202.txt When there are a few blacklisted nodes in the cluster, bin/mapred job -list-active-trackers throws java.lang.ClassCastException: org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200758#comment-13200758 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Mapreduce-0.23-Build #182 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/182/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) - Merging r1240603 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240604 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java Change MiniYarnCluster to escape special chars in testname -- Key: MAPREDUCE-3775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MR-3775.1.patch When using MiniYarnCluster with the testname set to a nested classname, the $ within the class name creates issues with the container launch scripts as they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200760#comment-13200760 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Mapreduce-0.23-Build #182 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/182/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) - Merging r1240634 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240635 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting Key: MAPREDUCE-3765 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch FifoScheduler uses default min 1 GB regardless of the configuration value set for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200759#comment-13200759
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Mapreduce-0.23-Build #182 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/182/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev) -
Merging r1240587 from trunk
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240588
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site/site.xml
can't build site in hadoop-yarn-server-common
-
Key: MAPREDUCE-3791
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: documentation, mrv2
Affects Versions: 0.23.1
Reporter: Roman Shaposhnik
Assignee: Mahadev konar
Fix For: 0.23.1
Attachments: MAPREDUCE-3791.patch
Here's how to reproduce:
{noformat}
$ mvn site site:stage -DskipTests -DskipTest -DskipITs
main:
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main SUCCESS [49.017s]
[INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
[INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
[INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
[INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
[INFO] Apache Hadoop Auth SUCCESS [5.215s]
[INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
[INFO] Apache Hadoop Common .. SUCCESS [4.111s]
[INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
[INFO] Apache Hadoop HDFS SUCCESS [5.373s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
[INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
[INFO] hadoop-yarn ... SUCCESS [5.250s]
[INFO] hadoop-yarn-api ... SUCCESS [4.579s]
[INFO] hadoop-yarn-common SUCCESS [4.268s]
[INFO] hadoop-yarn-server SUCCESS [4.408s]
[INFO] hadoop-yarn-server-common . FAILURE [0.035s]
[INFO] hadoop-yarn-server-nodemanager SKIPPED
[INFO] hadoop-yarn-server-web-proxy .. SKIPPED
[INFO] hadoop-yarn-server-resourcemanager SKIPPED
[INFO] hadoop-yarn-server-tests .. SKIPPED
[INFO] hadoop-mapreduce-client ... SKIPPED
[INFO] hadoop-mapreduce-client-core .. SKIPPED
[INFO] hadoop-yarn-applications .. SKIPPED
[INFO] hadoop-yarn-applications-distributedshell . SKIPPED
[INFO] hadoop-yarn-site .. SKIPPED
[INFO] hadoop-mapreduce-client-common SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
[INFO] hadoop-mapreduce-client-app ... SKIPPED
[INFO] hadoop-mapreduce-client-hs SKIPPED
[INFO] hadoop-mapreduce-client-jobclient . SKIPPED
[INFO] Apache Hadoop MapReduce Examples .. SKIPPED
[INFO] hadoop-mapreduce .. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming . SKIPPED
[INFO] Apache Hadoop Distributed Copy SKIPPED
[INFO] Apache Hadoop Archives SKIPPED
[INFO] Apache Hadoop Rumen ... SKIPPED
[INFO] Apache Hadoop Extras .. SKIPPED
[INFO] Apache Hadoop Tools Dist .. SKIPPED
[INFO] Apache Hadoop Tools ... SKIPPED
[INFO] Apache Hadoop Distribution SKIPPED
[INFO] Apache Hadoop Client .. SKIPPED
[INFO] Apache Hadoop Mini-Cluster SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 2:18.427s
[INFO] Finished
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200761#comment-13200761
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Mapreduce-0.23-Build #182 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/182/])
MAPREDUCE-3723. TestAMWebServicesJobs TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) - Merging r1240590
from trunk.
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240591
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
TestAMWebServicesJobs TestHSWebServicesJobs incorrectly asserting tests
-
Key: MAPREDUCE-3723
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, test, webapps
Affects Versions: 0.23.1, 0.24.0
Reporter: Bhallamudi Venkata Siva Kamesh
Assignee: Bhallamudi Venkata Siva Kamesh
Fix For: 0.23.1
Attachments: MAPREDUCE-3723.patch
While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
TestHSWebServicesJobs incorrectly asserting tests.
Moreover tests may fail if
{noformat}
index of counterGroups #counters in a particular counterGroup
{noformat}
{code:title=TestAMWebServicesJobs.java|borderStyle=solid}
for (int j = 0; j counters.length(); j++) {
JSONObject counter = counters.getJSONObject(i);
{code}
where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) job -status command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200774#comment-13200774 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Mapreduce-trunk #980 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/980/]) MAPREDUCE-3795. job -status command line output is malformed. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240593 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java job -status command line output is malformed -- Key: MAPREDUCE-3795 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3795-20120202.txt Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200775#comment-13200775 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Mapreduce-trunk #980 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/980/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240598 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes --- Key: MAPREDUCE-3759 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3759-20120202.txt When there are a few blacklisted nodes in the cluster, bin/mapred job -list-active-trackers throws java.lang.ClassCastException: org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200778#comment-13200778
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Mapreduce-trunk #980 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/980/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240587
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/trunk/hadoop-project/src/site/site.xml
can't build site in hadoop-yarn-server-common
-
Key: MAPREDUCE-3791
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: documentation, mrv2
Affects Versions: 0.23.1
Reporter: Roman Shaposhnik
Assignee: Mahadev konar
Fix For: 0.23.1
Attachments: MAPREDUCE-3791.patch
Here's how to reproduce:
{noformat}
$ mvn site site:stage -DskipTests -DskipTest -DskipITs
main:
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main SUCCESS [49.017s]
[INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
[INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
[INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
[INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
[INFO] Apache Hadoop Auth SUCCESS [5.215s]
[INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
[INFO] Apache Hadoop Common .. SUCCESS [4.111s]
[INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
[INFO] Apache Hadoop HDFS SUCCESS [5.373s]
[INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
[INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
[INFO] hadoop-yarn ... SUCCESS [5.250s]
[INFO] hadoop-yarn-api ... SUCCESS [4.579s]
[INFO] hadoop-yarn-common SUCCESS [4.268s]
[INFO] hadoop-yarn-server SUCCESS [4.408s]
[INFO] hadoop-yarn-server-common . FAILURE [0.035s]
[INFO] hadoop-yarn-server-nodemanager SKIPPED
[INFO] hadoop-yarn-server-web-proxy .. SKIPPED
[INFO] hadoop-yarn-server-resourcemanager SKIPPED
[INFO] hadoop-yarn-server-tests .. SKIPPED
[INFO] hadoop-mapreduce-client ... SKIPPED
[INFO] hadoop-mapreduce-client-core .. SKIPPED
[INFO] hadoop-yarn-applications .. SKIPPED
[INFO] hadoop-yarn-applications-distributedshell . SKIPPED
[INFO] hadoop-yarn-site .. SKIPPED
[INFO] hadoop-mapreduce-client-common SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
[INFO] hadoop-mapreduce-client-app ... SKIPPED
[INFO] hadoop-mapreduce-client-hs SKIPPED
[INFO] hadoop-mapreduce-client-jobclient . SKIPPED
[INFO] Apache Hadoop MapReduce Examples .. SKIPPED
[INFO] hadoop-mapreduce .. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming . SKIPPED
[INFO] Apache Hadoop Distributed Copy SKIPPED
[INFO] Apache Hadoop Archives SKIPPED
[INFO] Apache Hadoop Rumen ... SKIPPED
[INFO] Apache Hadoop Extras .. SKIPPED
[INFO] Apache Hadoop Tools Dist .. SKIPPED
[INFO] Apache Hadoop Tools ... SKIPPED
[INFO] Apache Hadoop Distribution SKIPPED
[INFO] Apache Hadoop Client .. SKIPPED
[INFO] Apache Hadoop Mini-Cluster SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 2:18.427s
[INFO] Finished at: Thu Feb 02 10:31:35 PST 2012
[INFO] Final Memory: 321M/1012M
[INFO]
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200776#comment-13200776 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Mapreduce-trunk #980 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/980/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240603 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java Change MiniYarnCluster to escape special chars in testname -- Key: MAPREDUCE-3775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MR-3775.1.patch When using MiniYarnCluster with the testname set to a nested classname, the $ within the class name creates issues with the container launch scripts as they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200779#comment-13200779 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Mapreduce-trunk #980 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/980/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240634 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting Key: MAPREDUCE-3765 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch FifoScheduler uses default min 1 GB regardless of the configuration value set for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200780#comment-13200780
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Mapreduce-trunk #980 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/980/])
MAPREDUCE-3723. TestAMWebServicesJobs TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240590
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
TestAMWebServicesJobs TestHSWebServicesJobs incorrectly asserting tests
-
Key: MAPREDUCE-3723
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, test, webapps
Affects Versions: 0.23.1, 0.24.0
Reporter: Bhallamudi Venkata Siva Kamesh
Assignee: Bhallamudi Venkata Siva Kamesh
Fix For: 0.23.1
Attachments: MAPREDUCE-3723.patch
While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
TestHSWebServicesJobs incorrectly asserting tests.
Moreover tests may fail if
{noformat}
index of counterGroups #counters in a particular counterGroup
{noformat}
{code:title=TestAMWebServicesJobs.java|borderStyle=solid}
for (int j = 0; j counters.length(); j++) {
JSONObject counter = counters.getJSONObject(i);
{code}
where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3809) Tasks may take upto 3 seconds to exit after completion
Tasks may take upto 3 seconds to exit after completion -- Key: MAPREDUCE-3809 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3809 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 1.0.0, 0.23.1 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Blocker Task.TaskReporter.stopCommunicationThread can end up waiting for a thread.sleep(3000) before stopping the thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3809) Tasks may take upto 3 seconds to exit after completion
[ https://issues.apache.org/jira/browse/MAPREDUCE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-3809: -- Attachment: MR3809.txt Patch changes the sleep to a wait, and added a notify in stopCommunicationThread. Verified that the patch gets rid of the unnecessary wait on a local cluster. Tasks may take upto 3 seconds to exit after completion -- Key: MAPREDUCE-3809 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3809 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.1, 1.0.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Blocker Attachments: MR3809.txt Task.TaskReporter.stopCommunicationThread can end up waiting for a thread.sleep(3000) before stopping the thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3809) Tasks may take upto 3 seconds to exit after completion
[ https://issues.apache.org/jira/browse/MAPREDUCE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-3809: -- Status: Patch Available (was: Open) Tasks may take upto 3 seconds to exit after completion -- Key: MAPREDUCE-3809 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3809 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 1.0.0, 0.23.1 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Blocker Attachments: MR3809.txt Task.TaskReporter.stopCommunicationThread can end up waiting for a thread.sleep(3000) before stopping the thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3810) MR AM's ContainerAllocator is assigning the allocated containers very slowly
[ https://issues.apache.org/jira/browse/MAPREDUCE-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3810: --- Description: This is mostly due to logging and other not-so-cheap operations we are doing as part of the AM-RM heartbeat cycle. Priority: Blocker (was: Major) Affects Version/s: 0.23.0 Fix Version/s: 0.23.1 Assignee: Vinod Kumar Vavilapalli MR AM's ContainerAllocator is assigning the allocated containers very slowly Key: MAPREDUCE-3810 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3810 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Priority: Blocker Fix For: 0.23.1 This is mostly due to logging and other not-so-cheap operations we are doing as part of the AM-RM heartbeat cycle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3810) MR AM's ContainerAllocator is assigning the allocated containers very slowly
MR AM's ContainerAllocator is assigning the allocated containers very slowly Key: MAPREDUCE-3810 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3810 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3809) Tasks may take upto 3 seconds to exit after completion
[ https://issues.apache.org/jira/browse/MAPREDUCE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200908#comment-13200908 ] Hadoop QA commented on MAPREDUCE-3809: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513363/MR3809.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1779//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1779//console This message is automatically generated. Tasks may take upto 3 seconds to exit after completion -- Key: MAPREDUCE-3809 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3809 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.1, 1.0.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Blocker Attachments: MR3809.txt Task.TaskReporter.stopCommunicationThread can end up waiting for a thread.sleep(3000) before stopping the thread. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3811) Make the Client-AM IPC retry count configurable
Make the Client-AM IPC retry count configurable --- Key: MAPREDUCE-3811 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3811 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Critical -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3811) Make the Client-AM IPC retry count configurable
[ https://issues.apache.org/jira/browse/MAPREDUCE-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-3811: -- Attachment: MR3811.txt Trivial patch. Make the Client-AM IPC retry count configurable --- Key: MAPREDUCE-3811 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3811 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Critical Attachments: MR3811.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3811) Make the Client-AM IPC retry count configurable
[ https://issues.apache.org/jira/browse/MAPREDUCE-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-3811: -- Status: Patch Available (was: Open) Make the Client-AM IPC retry count configurable --- Key: MAPREDUCE-3811 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3811 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Critical Attachments: MR3811.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3812) Change default AM slot size to be 1GB
Change default AM slot size to be 1GB - Key: MAPREDUCE-3812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3812) Change default AM slot size to be 1GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3812: --- Description: After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB. This is just a configuration change. Affects Version/s: 0.23.0 Fix Version/s: 0.23.1 Assignee: Vinod Kumar Vavilapalli Change default AM slot size to be 1GB - Key: MAPREDUCE-3812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB. This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3813) RackResolver should maintain a cache to avoid repetitive lookups.
RackResolver should maintain a cache to avoid repetitive lookups. - Key: MAPREDUCE-3813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3813 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3813) RackResolver should maintain a cache to avoid repetitive lookups.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3813: --- Description: With the current code, during task creation, we repeatedly resolve hosts and RackResolver doesn't cache any of the results. Caching will improve performance. Affects Version/s: 0.23.0 Fix Version/s: 0.23.1 Assignee: Vinod Kumar Vavilapalli RackResolver should maintain a cache to avoid repetitive lookups. - Key: MAPREDUCE-3813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3813 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 With the current code, during task creation, we repeatedly resolve hosts and RackResolver doesn't cache any of the results. Caching will improve performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3811) Make the Client-AM IPC retry count configurable
[ https://issues.apache.org/jira/browse/MAPREDUCE-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200950#comment-13200950 ] Hadoop QA commented on MAPREDUCE-3811: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513371/MR3811.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1780//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1780//console This message is automatically generated. Make the Client-AM IPC retry count configurable --- Key: MAPREDUCE-3811 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3811 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Critical Attachments: MR3811.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3697) Hadoop Counters API limits Oozie's working across different hadoop versions
[ https://issues.apache.org/jira/browse/MAPREDUCE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200955#comment-13200955 ] Mahadev konar commented on MAPREDUCE-3697: -- Looks like there is an existing race condition in Counters that is showing up with this patch. I am still hunting it down. TestJobCounters fails intermittently on trunk even without the patch. Hadoop Counters API limits Oozie's working across different hadoop versions --- Key: MAPREDUCE-3697 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3697 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: John George Assignee: Mahadev konar Priority: Blocker Attachments: MAPREDUCE-3697-facade.patch, MAPREDUCE-3697-facadeV2.patch, MAPREDUCE-3697-facadeV3.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its parent class AbstractCounters implements it. This is different from hadoop20X. As a result, Oozie compiled with either hadoop version does not work with the other version. A specific scenario, Oozie compiled with .23 and run against 205, does not update job status owing to a Counters API exception. Will explicit re-compilation against the relevant hadoop jars be required each time? This will prevent launching a uniform Oozie version across different clusters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3812: --- Summary: Change default memory slot sizes to be 1.5GB (was: Change default AM slot size to be 1GB) Digging more, I realized we've been testing with 1.5 slot size for normal tasks and the same 1.5GB for AM since a long time. I am going to change the defaults to reflect what we've been testing with. Change default memory slot sizes to be 1.5GB Key: MAPREDUCE-3812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB. This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3812: --- Attachment: MAPREDUCE-3812-20120205.txt - Changed default minimum-allocation-mb to be 1536, maximum-allocation-mb to be 6*1536, default total memory per node to be 6*1536. - Changed default slot size for AM to be 1536, heap size to be 1024mb. - Also removed the per-scheduler min and max allocation mb configurations. These are common concepts across schedulers. Change default memory slot sizes to be 1.5GB Key: MAPREDUCE-3812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3812-20120205.txt After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB. This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3812: --- Status: Patch Available (was: Open) Change default memory slot sizes to be 1.5GB Key: MAPREDUCE-3812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3812-20120205.txt After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB. This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200976#comment-13200976 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1781//console This message is automatically generated. Change default memory slot sizes to be 1.5GB Key: MAPREDUCE-3812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3812-20120205.txt After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB. This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200978#comment-13200978 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1782//console This message is automatically generated. Change default memory slot sizes to be 1.5GB Key: MAPREDUCE-3812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3812-20120205.txt After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work around the issues with AM heap. Now that those are fixed, we should go back to 1GB. This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200989#comment-13200989 ] Mahadev konar commented on MAPREDUCE-3747: -- The patch looks good. We should re run it through hudson just to make sure that the units tests pass. Memory Total is not refreshed until an app is launched -- Key: MAPREDUCE-3747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch Memory Total on the RM UI is not refreshed until an application is launched. This is a problem when the cluster is started for the first time or when there are any lost/decommissioned NMs. When the cluster is started for the first time, Active Nodes is 0 but the Memory Total=0. Also when there are any lost/decommissioned nodes, Memory Total has wrong value. This is a useful tool for cluster admins and has to be updated correctly without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3810) MR AM's ContainerAllocator is assigning the allocated containers very slowly
[ https://issues.apache.org/jira/browse/MAPREDUCE-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3810: --- Attachment: MAPREDUCE-3810-20120205.txt This should fix it. Been using it to benchmark scan and it helped some improvement. MR AM's ContainerAllocator is assigning the allocated containers very slowly Key: MAPREDUCE-3810 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3810 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3810-20120205.txt This is mostly due to logging and other not-so-cheap operations we are doing as part of the AM-RM heartbeat cycle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3794) Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility
[ https://issues.apache.org/jira/browse/MAPREDUCE-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3794: - Component/s: mrv2 Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility Key: MAPREDUCE-3794 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3794 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Tom White Assignee: Tom White Attachments: MAPREDUCE-3794.patch The new counters are mapreduce.TaskCounter and mapreduce.JobCounter, but we should support the old ones too since they are public in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3810) MR AM's ContainerAllocator is assigning the allocated containers very slowly
[ https://issues.apache.org/jira/browse/MAPREDUCE-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3810: --- Status: Patch Available (was: Open) Patch summary: - Mostly moving log statements in the heartbeat critical path to debug level - Added equals/hashCode for priority for faster lookup (the default impl uses getProto() which becomes costly) and improved comparesTo() in ResourceRequest. MR AM's ContainerAllocator is assigning the allocated containers very slowly Key: MAPREDUCE-3810 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3810 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3810-20120205.txt This is mostly due to logging and other not-so-cheap operations we are doing as part of the AM-RM heartbeat cycle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3813) RackResolver should maintain a cache to avoid repetitive lookups.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3813: --- Attachment: MAPREDUCE-3813-20120205.txt This should fix it. Also, added a simple test. RackResolver should maintain a cache to avoid repetitive lookups. - Key: MAPREDUCE-3813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3813 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3813-20120205.txt With the current code, during task creation, we repeatedly resolve hosts and RackResolver doesn't cache any of the results. Caching will improve performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3813) RackResolver should maintain a cache to avoid repetitive lookups.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3813: --- Status: Patch Available (was: Open) RackResolver should maintain a cache to avoid repetitive lookups. - Key: MAPREDUCE-3813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3813 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3813-20120205.txt With the current code, during task creation, we repeatedly resolve hosts and RackResolver doesn't cache any of the results. Caching will improve performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201001#comment-13201001 ] Hadoop QA commented on MAPREDUCE-3747: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513274/MAPREDUCE-3747.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1783//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1783//console This message is automatically generated. Memory Total is not refreshed until an app is launched -- Key: MAPREDUCE-3747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch Memory Total on the RM UI is not refreshed until an application is launched. This is a problem when the cluster is started for the first time or when there are any lost/decommissioned NMs. When the cluster is started for the first time, Active Nodes is 0 but the Memory Total=0. Also when there are any lost/decommissioned nodes, Memory Total has wrong value. This is a useful tool for cluster admins and has to be updated correctly without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3810) MR AM's ContainerAllocator is assigning the allocated containers very slowly
[ https://issues.apache.org/jira/browse/MAPREDUCE-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201008#comment-13201008 ] Hadoop QA commented on MAPREDUCE-3810: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513383/MAPREDUCE-3810-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1784//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1784//console This message is automatically generated. MR AM's ContainerAllocator is assigning the allocated containers very slowly Key: MAPREDUCE-3810 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3810 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3810-20120205.txt This is mostly due to logging and other not-so-cheap operations we are doing as part of the AM-RM heartbeat cycle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3813) RackResolver should maintain a cache to avoid repetitive lookups.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201011#comment-13201011 ] Hadoop QA commented on MAPREDUCE-3813: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513387/MAPREDUCE-3813-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1785//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1785//console This message is automatically generated. RackResolver should maintain a cache to avoid repetitive lookups. - Key: MAPREDUCE-3813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3813 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3813-20120205.txt With the current code, during task creation, we repeatedly resolve hosts and RackResolver doesn't cache any of the results. Caching will improve performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3747: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Memory Total is not refreshed until an app is launched -- Key: MAPREDUCE-3747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch Memory Total on the RM UI is not refreshed until an application is launched. This is a problem when the cluster is started for the first time or when there are any lost/decommissioned NMs. When the cluster is started for the first time, Active Nodes is 0 but the Memory Total=0. Also when there are any lost/decommissioned nodes, Memory Total has wrong value. This is a useful tool for cluster admins and has to be updated correctly without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3814) MR1 compile fails
MR1 compile fails - Key: MAPREDUCE-3814 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3814 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.1 Reporter: Arun C Murthy $ ant veryclean all-jars -Dversion=0.23.1 -Dresolvers=internal BUILD FAILED /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/build.xml:537: srcdir /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/src/test/mapred/testjar does not exist! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3746) Nodemanagers are not automatically shut down after decommissioning
[ https://issues.apache.org/jira/browse/MAPREDUCE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201016#comment-13201016 ] Hudson commented on MAPREDUCE-3746: --- Integrated in Hadoop-Hdfs-0.23-Commit #483 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/483/]) Merge -c 1240886 from trunk to branch-0.23 to fix MAPREDUCE-3746. Initialize queue metrics upfront and added start/finish time to RM Web-UI. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240887 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/UserMetricsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java Nodemanagers are not automatically shut down after decommissioning -- Key: MAPREDUCE-3746 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3746 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Devaraj K Priority: Critical Fix For: 0.23.1 Nodemanagers are not automatically shutdown after decommissioning. MAPREDUCE-2775 does not seem to fix the issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201017#comment-13201017 ] Hudson commented on MAPREDUCE-3747: --- Integrated in Hadoop-Hdfs-trunk-Commit #1738 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1738/]) MAPREDUCE-3747. Initialize queue metrics upfront and added start/finish time to RM Web-UI. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240886 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/UserMetricsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java Memory Total is not refreshed until an app is launched -- Key: MAPREDUCE-3747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch Memory Total on the RM UI is not refreshed until an application is launched. This is a problem when the cluster is started for the first time or when there are any lost/decommissioned NMs. When the cluster is started for the first time, Active Nodes is 0 but the Memory Total=0. Also when there are any lost/decommissioned nodes, Memory Total has wrong value. This is a useful tool for cluster admins and has to be updated correctly without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201018#comment-13201018 ] Hudson commented on MAPREDUCE-3747: --- Integrated in Hadoop-Common-trunk-Commit #1666 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1666/]) MAPREDUCE-3747. Initialize queue metrics upfront and added start/finish time to RM Web-UI. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240886 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/UserMetricsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java Memory Total is not refreshed until an app is launched -- Key: MAPREDUCE-3747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch Memory Total on the RM UI is not refreshed until an application is launched. This is a problem when the cluster is started for the first time or when there are any lost/decommissioned NMs. When the cluster is started for the first time, Active Nodes is 0 but the Memory Total=0. Also when there are any lost/decommissioned nodes, Memory Total has wrong value. This is a useful tool for cluster admins and has to be updated correctly without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3746) Nodemanagers are not automatically shut down after decommissioning
[ https://issues.apache.org/jira/browse/MAPREDUCE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201020#comment-13201020 ] Hudson commented on MAPREDUCE-3746: --- Integrated in Hadoop-Common-0.23-Commit #493 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/493/]) Merge -c 1240886 from trunk to branch-0.23 to fix MAPREDUCE-3746. Initialize queue metrics upfront and added start/finish time to RM Web-UI. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240887 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/UserMetricsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java Nodemanagers are not automatically shut down after decommissioning -- Key: MAPREDUCE-3746 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3746 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Devaraj K Priority: Critical Fix For: 0.23.1 Nodemanagers are not automatically shutdown after decommissioning. MAPREDUCE-2775 does not seem to fix the issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3814) MR1 compile fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3814: - Attachment: MAPREDUCE-3814.patch MAPREDUCE-3582 moved testjar testshell to hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java but missed updating build.xml. MR1 compile fails - Key: MAPREDUCE-3814 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3814 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 0.23.1 Reporter: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3814.patch $ ant veryclean all-jars -Dversion=0.23.1 -Dresolvers=internal BUILD FAILED /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/build.xml:537: srcdir /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/src/test/mapred/testjar does not exist! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3814) MR1 compile fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3814: - Component/s: mrv2 mrv1 MR1 compile fails - Key: MAPREDUCE-3814 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3814 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 0.23.1 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3814.patch $ ant veryclean all-jars -Dversion=0.23.1 -Dresolvers=internal BUILD FAILED /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/build.xml:537: srcdir /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/src/test/mapred/testjar does not exist! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3814) MR1 compile fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3814: - Fix Version/s: 0.23.1 Assignee: Arun C Murthy Status: Patch Available (was: Open) MR1 compile fails - Key: MAPREDUCE-3814 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3814 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 0.23.1 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3814.patch $ ant veryclean all-jars -Dversion=0.23.1 -Dresolvers=internal BUILD FAILED /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/build.xml:537: srcdir /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/src/test/mapred/testjar does not exist! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3815) Data Locality suffers if HDFS returns IPs in getFileBlockLocations
Data Locality suffers if HDFS returns IPs in getFileBlockLocations -- Key: MAPREDUCE-3815 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3815 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth Priority: Critical BlockLocation.getHosts() returns IP addresses occasionally. Data locality is affected - since the RM requires hostnames. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3815) Data Locality suffers if HDFS returns IPs in getFileBlockLocations
[ https://issues.apache.org/jira/browse/MAPREDUCE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth reassigned MAPREDUCE-3815: - Assignee: Siddharth Seth Data Locality suffers if HDFS returns IPs in getFileBlockLocations -- Key: MAPREDUCE-3815 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3815 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Critical BlockLocation.getHosts() returns IP addresses occasionally. Data locality is affected - since the RM requires hostnames. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3746) Nodemanagers are not automatically shut down after decommissioning
[ https://issues.apache.org/jira/browse/MAPREDUCE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201028#comment-13201028 ] Hudson commented on MAPREDUCE-3746: --- Integrated in Hadoop-Mapreduce-0.23-Commit #504 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/504/]) Merge -c 1240886 from trunk to branch-0.23 to fix MAPREDUCE-3746. Initialize queue metrics upfront and added start/finish time to RM Web-UI. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240887 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/UserMetricsInfo.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java Nodemanagers are not automatically shut down after decommissioning -- Key: MAPREDUCE-3746 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3746 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Devaraj K Priority: Critical Fix For: 0.23.1 Nodemanagers are not automatically shutdown after decommissioning. MAPREDUCE-2775 does not seem to fix the issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201027#comment-13201027 ] Hudson commented on MAPREDUCE-3747: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1681 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1681/]) MAPREDUCE-3747. Initialize queue metrics upfront and added start/finish time to RM Web-UI. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1240886 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/UserMetricsInfo.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java Memory Total is not refreshed until an app is launched -- Key: MAPREDUCE-3747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch Memory Total on the RM UI is not refreshed until an application is launched. This is a problem when the cluster is started for the first time or when there are any lost/decommissioned NMs. When the cluster is started for the first time, Active Nodes is 0 but the Memory Total=0. Also when there are any lost/decommissioned nodes, Memory Total has wrong value. This is a useful tool for cluster admins and has to be updated correctly without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3814) MR1 compile fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201038#comment-13201038 ] Hadoop QA commented on MAPREDUCE-3814: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513392/MAPREDUCE-3814.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1786//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1786//console This message is automatically generated. MR1 compile fails - Key: MAPREDUCE-3814 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3814 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 0.23.1 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3814.patch $ ant veryclean all-jars -Dversion=0.23.1 -Dresolvers=internal BUILD FAILED /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/build.xml:537: srcdir /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/src/test/mapred/testjar does not exist! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3813) RackResolver should maintain a cache to avoid repetitive lookups.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201046#comment-13201046 ] Allen Wittenauer commented on MAPREDUCE-3813: - If this is to cache hostname resolution, this shouldn't be necessary as Java already does this for you for free. See the discussion under HDFS-34. RackResolver should maintain a cache to avoid repetitive lookups. - Key: MAPREDUCE-3813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3813 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3813-20120205.txt With the current code, during task creation, we repeatedly resolve hosts and RackResolver doesn't cache any of the results. Caching will improve performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3813) RackResolver should maintain a cache to avoid repetitive lookups.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201052#comment-13201052 ] Allen Wittenauer commented on MAPREDUCE-3813: - Woops, also meant to point out that properly configured hosts have an OS-level cache for hostnames. So unless there are some benchmarks to back up the claims, I'm not convinced that hostname resolution should be an issue (with the caveat that the machines should be configured correctly). RackResolver should maintain a cache to avoid repetitive lookups. - Key: MAPREDUCE-3813 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3813 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, performance Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 0.23.1 Attachments: MAPREDUCE-3813-20120205.txt With the current code, during task creation, we repeatedly resolve hosts and RackResolver doesn't cache any of the results. Caching will improve performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201053#comment-13201053
]
Ahmed Radwan commented on MAPREDUCE-3436:
-
@Mahadev,
There are two issues discussed here (please see my previous comment for
description of the two issues). The current patch addresses issue#1.
I am planning also on looking into how to resolve issue#2 which Siddharth
highlighted. I was busy so didn't have time to look into it, but planning to do
so in the coming few days.
So if there is urgency in pushing the current patch (which seems to be accepted
by Siddharth and Bruno), I can create another ticket to follow issue#2. If
there is no urgency then I'll investigate issue#2 in the coming days and update
the current patch. Thanks!
jobhistory link may be broken depending on the interface it is listening on
---
Key: MAPREDUCE-3436
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, webapps
Affects Versions: 0.23.0, 0.23.1
Reporter: Bruno Mahé
Assignee: Ahmed Radwan
Labels: bigtop
Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
On the following page : http://RESOURCE_MANAGER:8088/cluster/apps
There are links to the history for each application. None of them can be
reached since they all point to the ip 0.0.0.0. For instance:
http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
Am I missing something?
[root@bigtop-fedora-15 ~]# jps
9968 ResourceManager
1495 NameNode
1645 DataNode
12935 Jps
11140 -- process information unavailable
5309 JobHistoryServer
10237 NodeManager
[root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
tcp0 0 :::8088 :::*
LISTEN 9968/java
For reference, here is my configuration:
root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
?xml version=1.0?
configuration
!-- Site specific YARN configuration properties --
property
nameyarn.nodemanager.aux-services/name
valuemapreduce.shuffle/value
/property
property
nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
valueorg.apache.hadoop.mapred.ShuffleHandler/value
/property
property
namemapreduce.admin.user.env/name
valueCLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*/value
/property
/configuration
[root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
?xml version=1.0?
configuration
property
namedfs.replication/name
value1/value
/property
property
namedfs.permissions/name
valuefalse/value
/property
property
!-- specify this so that running 'hadoop namenode -format' formats the
right dir --
namedfs.name.dir/name
value/var/lib/hadoop/cache/hadoop/dfs/name/value
/property
/configuration
[root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
?xml version=1.0?
configuration
property
namefs.default.name/name
valuehdfs://localhost:8020/value
/property
property
namehadoop.tmp.dir/name
value/var/lib/hadoop/cache/${user.name}/value
/property
!-- OOZIE proxy user setting --
property
namehadoop.proxyuser.oozie.hosts/name
value*/value
/property
property
namehadoop.proxyuser.oozie.groups/name
value*/value
/property
/configuration
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3794) Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility
[ https://issues.apache.org/jira/browse/MAPREDUCE-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201055#comment-13201055 ] Mahadev konar commented on MAPREDUCE-3794: -- @Tom, Thanks for doing this. Looks like a compilation error in the patch. Mind updating it? Good one to have in 0.23.1. Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility Key: MAPREDUCE-3794 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3794 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Tom White Assignee: Tom White Attachments: MAPREDUCE-3794.patch The new counters are mapreduce.TaskCounter and mapreduce.JobCounter, but we should support the old ones too since they are public in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3814) MR1 compile fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201069#comment-13201069 ] Mahadev konar commented on MAPREDUCE-3814: -- +1 the patch looks good to me. MR1 compile fails - Key: MAPREDUCE-3814 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3814 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 0.23.1 Reporter: Arun C Murthy Assignee: Arun C Murthy Fix For: 0.23.1 Attachments: MAPREDUCE-3814.patch $ ant veryclean all-jars -Dversion=0.23.1 -Dresolvers=internal BUILD FAILED /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/build.xml:537: srcdir /grid/0/dev/acm/hadoop-0.23/hadoop-mapreduce-project/src/test/mapred/testjar does not exist! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3815) Data Locality suffers if HDFS returns IPs in getFileBlockLocations
[ https://issues.apache.org/jira/browse/MAPREDUCE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201080#comment-13201080 ] Eli Collins commented on MAPREDUCE-3815: This should be an HDFS jira right? Data Locality suffers if HDFS returns IPs in getFileBlockLocations -- Key: MAPREDUCE-3815 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3815 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth Assignee: Siddharth Seth Priority: Critical BlockLocation.getHosts() returns IP addresses occasionally. Data locality is affected - since the RM requires hostnames. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
