[jira] [Assigned] (MAPREDUCE-3642) Remove hardcoded strings from the JC#displayTasks() call.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J reassigned MAPREDUCE-3642: -- Assignee: (was: Harsh J) > Remove hardcoded strings from the JC#displayTasks() call. > - > > Key: MAPREDUCE-3642 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3642 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Affects Versions: 0.24.0 >Reporter: Harsh J > Labels: newbie > > This is to address Eli's comments on the parent task: > bq. 1. The error messages should generate the lists of valid states and types > from their definitions rather than hard-coding them into the error messages. > bq. 2. Aren't these types and states defined somewhere already? Seems like > they're a public API and therefore shouldn't have to duplicate the definition > of them in taskTypes and taskStates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200628#comment-13200628 ] Arun C Murthy commented on MAPREDUCE-3747: -- TestJobCounters failure is unrelated. > Memory Total is not refreshed until an app is launched > -- > > Key: MAPREDUCE-3747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Arun C Murthy > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, > MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch > > > Memory Total on the RM UI is not refreshed until an application is launched. > This is a problem when the cluster is started for the first time or when > there are any lost/decommissioned NMs. > When the cluster is started for the first time, Active Nodes is > 0 but the > Memory Total=0. Also when there are any lost/decommissioned nodes, Memory > Total has wrong value. > This is a useful tool for cluster admins and has to be updated correctly > without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3709) TestDistributedShell is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reassigned MAPREDUCE-3709: -- Assignee: Hitesh Shah > TestDistributedShell is failing > --- > > Key: MAPREDUCE-3709 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3709 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, test >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Hitesh Shah >Priority: Blocker > > TestDistributedShell#testDSShell is failing the assert on line 90 on > branch-23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200624#comment-13200624 ] Hadoop QA commented on MAPREDUCE-3747: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513274/MAPREDUCE-3747.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestJobCounters +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1778//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1778//console This message is automatically generated. > Memory Total is not refreshed until an app is launched > -- > > Key: MAPREDUCE-3747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Arun C Murthy > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, > MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch > > > Memory Total on the RM UI is not refreshed until an application is launched. > This is a problem when the cluster is started for the first time or when > there are any lost/decommissioned NMs. > When the cluster is started for the first time, Active Nodes is > 0 but the > Memory Total=0. Also when there are any lost/decommissioned nodes, Memory > Total has wrong value. > This is a useful tool for cluster admins and has to be updated correctly > without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3747: - Attachment: MAPREDUCE-3747.patch Fixed. > Memory Total is not refreshed until an app is launched > -- > > Key: MAPREDUCE-3747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Arun C Murthy > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, > MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch > > > Memory Total on the RM UI is not refreshed until an application is launched. > This is a problem when the cluster is started for the first time or when > there are any lost/decommissioned NMs. > When the cluster is started for the first time, Active Nodes is > 0 but the > Memory Total=0. Also when there are any lost/decommissioned nodes, Memory > Total has wrong value. > This is a useful tool for cluster admins and has to be updated correctly > without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3747: - Status: Patch Available (was: Open) > Memory Total is not refreshed until an app is launched > -- > > Key: MAPREDUCE-3747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Arun C Murthy > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, > MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, MAPREDUCE-3747.patch > > > Memory Total on the RM UI is not refreshed until an application is launched. > This is a problem when the cluster is started for the first time or when > there are any lost/decommissioned NMs. > When the cluster is started for the first time, Active Nodes is > 0 but the > Memory Total=0. Also when there are any lost/decommissioned nodes, Memory > Total has wrong value. > This is a useful tool for cluster admins and has to be updated correctly > without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3747: - Status: Open (was: Patch Available) Gah, wrong patch > Memory Total is not refreshed until an app is launched > -- > > Key: MAPREDUCE-3747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Arun C Murthy > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, > MAPREDUCE-3747.patch, MAPREDUCE-3747.patch > > > Memory Total on the RM UI is not refreshed until an application is launched. > This is a problem when the cluster is started for the first time or when > there are any lost/decommissioned NMs. > When the cluster is started for the first time, Active Nodes is > 0 but the > Memory Total=0. Also when there are any lost/decommissioned nodes, Memory > Total has wrong value. > This is a useful tool for cluster admins and has to be updated correctly > without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3747: - Attachment: MAPREDUCE-3747.patch Completed now, fixed tests. > Memory Total is not refreshed until an app is launched > -- > > Key: MAPREDUCE-3747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Arun C Murthy > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, > MAPREDUCE-3747.patch, MAPREDUCE-3747.patch > > > Memory Total on the RM UI is not refreshed until an application is launched. > This is a problem when the cluster is started for the first time or when > there are any lost/decommissioned NMs. > When the cluster is started for the first time, Active Nodes is > 0 but the > Memory Total=0. Also when there are any lost/decommissioned nodes, Memory > Total has wrong value. > This is a useful tool for cluster admins and has to be updated correctly > without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
[ https://issues.apache.org/jira/browse/MAPREDUCE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3747: - Status: Patch Available (was: Open) > Memory Total is not refreshed until an app is launched > -- > > Key: MAPREDUCE-3747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Arun C Murthy > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3747.patch, MAPREDUCE-3747.patch, > MAPREDUCE-3747.patch, MAPREDUCE-3747.patch > > > Memory Total on the RM UI is not refreshed until an application is launched. > This is a problem when the cluster is started for the first time or when > there are any lost/decommissioned NMs. > When the cluster is started for the first time, Active Nodes is > 0 but the > Memory Total=0. Also when there are any lost/decommissioned nodes, Memory > Total has wrong value. > This is a useful tool for cluster admins and has to be updated correctly > without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200594#comment-13200594 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Mapreduce-0.23-Commit #503 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/503/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) - Merging r1240634 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240635 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200592#comment-13200592 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1679 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1679/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240634 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200586#comment-13200586 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Common-0.23-Commit #492 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/492/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) - Merging r1240634 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240635 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200585#comment-13200585 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Hdfs-0.23-Commit #482 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/482/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) - Merging r1240634 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240635 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200584#comment-13200584 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Common-trunk-Commit #1664 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1664/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240634 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200583#comment-13200583 ] Hudson commented on MAPREDUCE-3765: --- Integrated in Hadoop-Hdfs-trunk-Commit #1736 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1736/]) MAPREDUCE-3765. FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240634 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/dev-support/findbugs-exclude.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3765: - Resolution: Fixed Fix Version/s: 0.23.1 Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I just committed this. Thanks Hitesh! > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3761) AM info in job -list does not reflect the actual AM hostname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3761: - Status: Open (was: Patch Available) Vinod, Can you take a look at the test? > AM info in job -list does not reflect the actual AM hostname > > > Key: MAPREDUCE-3761 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3761 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3761-20120202.txt > > > The AM info field on "bin/mapred job -list" currently has a value > :8088/proxy/appID. This info is irrelevant unless > it shows the real information of where the AM was launched. This needs to be > fixed to show the AM host details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200564#comment-13200564 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1678 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1678/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240603 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200557#comment-13200557 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Mapreduce-0.23-Commit #502 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/502/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) - Merging r1240603 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240604 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200556#comment-13200556 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Mapreduce-0.23-Commit #502 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/502/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) - Merging r1240598 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240599 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200553#comment-13200553 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1677 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1677/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240598 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200550#comment-13200550 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Common-0.23-Commit #491 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/491/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) - Merging r1240603 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240604 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200549#comment-13200549 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Hdfs-0.23-Commit #481 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/481/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) - Merging r1240603 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240604 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200546#comment-13200546 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Common-trunk-Commit #1663 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1663/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240603 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200545#comment-13200545 ] Hudson commented on MAPREDUCE-3775: --- Integrated in Hadoop-Hdfs-trunk-Commit #1735 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1735/]) MAPREDUCE-3775. Change MiniYarnCluster to escape special chars in testname. (Hitesh Shah via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240603 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3775: - Resolution: Fixed Fix Version/s: 0.23.1 Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I just committed this. Thanks Hitesh! > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Fix For: 0.23.1 > > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200544#comment-13200544 ] Hitesh Shah commented on MAPREDUCE-3765: Yes - the patch is more or less same as the first one which threw a findbugs warning. The warning can be ignored as the variable is initialized only once and never changed by the scheduler. > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200540#comment-13200540 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Common-trunk-Commit #1662 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1662/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240598 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200541#comment-13200541 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Mapreduce-0.23-Commit #501 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/501/]) MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) - Merging r1240593 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240594 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200542#comment-13200542
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Mapreduce-0.23-Commit #501 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/501/])
MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) - Merging r1240590
from trunk.
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240591
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200539#comment-13200539 ] Mahadev konar commented on MAPREDUCE-3775: -- The test failure is due to MAPREDUCE-3583. Ill go ahead and commit this. Good change to have. > Change MiniYarnCluster to escape special chars in testname > -- > > Key: MAPREDUCE-3775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3775.1.patch > > > When using MiniYarnCluster with the testname set to a nested classname, the > "$" within the class name creates issues with the container launch scripts as > they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
[ https://issues.apache.org/jira/browse/MAPREDUCE-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200537#comment-13200537 ] Mahadev konar commented on MAPREDUCE-3765: -- @Hitesh, Is the findbugs exclude necessary with the latest patch? > FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb > setting > > > Key: MAPREDUCE-3765 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Minor > Attachments: MR-3765.1.patch, MR-3765.2.patch, MR-3765.3.patch > > > FifoScheduler uses default min 1 GB regardless of the configuration value set > for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200534#comment-13200534 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Hdfs-0.23-Commit #480 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/480/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) - Merging r1240598 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240599 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200536#comment-13200536 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Common-0.23-Commit #490 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/490/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) - Merging r1240598 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240599 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200535#comment-13200535 ] Hudson commented on MAPREDUCE-3759: --- Integrated in Hadoop-Hdfs-trunk-Commit #1734 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1734/]) MAPREDUCE-3759. ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240598 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3804) yarn webapp interface vulnerable to cross scripting attacks
[ https://issues.apache.org/jira/browse/MAPREDUCE-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3804: - Status: Open (was: Patch Available) @Dave, Can you please take a look at the findbugs issue? > yarn webapp interface vulnerable to cross scripting attacks > --- > > Key: MAPREDUCE-3804 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3804 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Dave Thompson >Assignee: Dave Thompson > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3804.patch, MAPREDUCE_3804_br_0.23.0.patch > > > Yarn webapp interface may be vulnerable to certain cross scripting attacks, > injected through URL request. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3761) AM info in job -list does not reflect the actual AM hostname
[ https://issues.apache.org/jira/browse/MAPREDUCE-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200530#comment-13200530 ] Hadoop QA commented on MAPREDUCE-3761: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513071/MAPREDUCE-3761-20120202.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapreduce.TestTypeConverter +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1777//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1777//console This message is automatically generated. > AM info in job -list does not reflect the actual AM hostname > > > Key: MAPREDUCE-3761 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3761 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3761-20120202.txt > > > The AM info field on "bin/mapred job -list" currently has a value > :8088/proxy/appID. This info is irrelevant unless > it shows the real information of where the AM was launched. This needs to be > fixed to show the AM host details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3759: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I just committed this. Thanks Vinod! > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
[ https://issues.apache.org/jira/browse/MAPREDUCE-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200527#comment-13200527 ] Mahadev konar commented on MAPREDUCE-3759: -- Patch looks good. Committing now. > ClassCastException thrown in -list-active-trackers when there are a few > unhealthy nodes > --- > > Key: MAPREDUCE-3759 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3759-20120202.txt > > > When there are a few blacklisted nodes in the cluster, "bin/mapred job > -list-active-trackers" throws "java.lang.ClassCastException: > org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be > cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200522#comment-13200522
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Mapreduce-trunk-Commit #1676 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1676/])
MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240590
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200521#comment-13200521 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1676 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1676/]) MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240593 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3808) NPE in FileOutputCommitter when running a 0 reduce job
NPE in FileOutputCommitter when running a 0 reduce job
--
Key: MAPREDUCE-3808
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Siddharth Seth
Priority: Blocker
This was while running LoadGen.
{noformat}
Error: java.lang.NullPointerException at
org.apache.hadoop.fs.Path.(Path.java:67)
at org.apache.hadoop.fs.Path.(Path.java:56)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
at
org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
at
org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
at
org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
at
org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
at org.apache.hadoop.mapred.Task.done(Task.java:912)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200520#comment-13200520 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Common-trunk-Commit #1661 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1661/]) MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240593 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200519#comment-13200519 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Common-0.23-Commit #489 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/489/]) MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) - Merging r1240593 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240594 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3761) AM info in job -list does not reflect the actual AM hostname
[
https://issues.apache.org/jira/browse/MAPREDUCE-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200518#comment-13200518
]
Mahadev konar commented on MAPREDUCE-3761:
--
Looks like the test failed because of:
{code}
ContainerMemoryManager while managing memory of container_0__01_00
java.lang.NumberFormatException: For input string: "18446743988089421650"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:422)
at java.lang.Long.parseLong(Long.java:468)
at
org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:427)
at
org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:168)
{code}
which is MAPREDUCE-3583. Running hudson again.
> AM info in job -list does not reflect the actual AM hostname
>
>
> Key: MAPREDUCE-3761
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3761
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Affects Versions: 0.23.1
>Reporter: Ramya Sunil
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3761-20120202.txt
>
>
> The AM info field on "bin/mapred job -list" currently has a value
> :8088/proxy/appID. This info is irrelevant unless
> it shows the real information of where the AM was launched. This needs to be
> fixed to show the AM host details.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200516#comment-13200516 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Hdfs-trunk-Commit #1733 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1733/]) MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240593 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200515#comment-13200515 ] Hudson commented on MAPREDUCE-3795: --- Integrated in Hadoop-Hdfs-0.23-Commit #479 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/479/]) MAPREDUCE-3795. "job -status" command line output is malformed. (vinodkv via mahadev) - Merging r1240593 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240594 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobMonitorAndPrint.java > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3697) Hadoop Counters API limits Oozie's working across different hadoop versions
[ https://issues.apache.org/jira/browse/MAPREDUCE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200513#comment-13200513 ] Hadoop QA commented on MAPREDUCE-3697: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513258/MAPREDUCE-3697.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestJobCounters +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1776//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1776//console This message is automatically generated. > Hadoop Counters API limits Oozie's working across different hadoop versions > --- > > Key: MAPREDUCE-3697 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3697 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: John George >Assignee: Mahadev konar >Priority: Blocker > Attachments: MAPREDUCE-3697-facade.patch, > MAPREDUCE-3697-facadeV2.patch, MAPREDUCE-3697-facadeV3.patch, > MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, > MAPREDUCE-3697.patch, MAPREDUCE-3697.patch > > > Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in > hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). > Its > parent class AbstractCounters implements it. This is different from hadoop20X. > As a result, Oozie compiled with either hadoop version does not work with the > other version. > A specific scenario, Oozie compiled with .23 and run against 205, does not > update job status owing to a Counters API exception. > Will explicit re-compilation against the relevant hadoop jars be required each > time? This will prevent launching a uniform Oozie version across different > clusters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3795: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Just committed this. Thanks Vinod! > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200511#comment-13200511
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Mapreduce-trunk-Commit #1675 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1675/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240587
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/trunk/hadoop-project/src/site/site.xml
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 2:18.427s
> [INFO] Finished at: Thu F
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200510#comment-13200510
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Mapreduce-0.23-Commit #500 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/500/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev) -
Merging r1240587 from trunk
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240588
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site/site.xml
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
[jira] [Commented] (MAPREDUCE-3795) "job -status" command line output is malformed
[ https://issues.apache.org/jira/browse/MAPREDUCE-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200509#comment-13200509 ] Mahadev konar commented on MAPREDUCE-3795: -- Patch looks good. Committing it now. > "job -status" command line output is malformed > -- > > Key: MAPREDUCE-3795 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3795 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3795-20120202.txt > > > Misses new lines after numMaps and numReduces. Caused by MAPREDUCE-3720. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3686) history server web ui - job counter values for map/reduce not shown properly
[ https://issues.apache.org/jira/browse/MAPREDUCE-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200508#comment-13200508 ] Mahadev konar commented on MAPREDUCE-3686: -- @Bhallamudi, Shouldnt you be using factory.newFrameworkGroup(int id), when the group is a framework group? > history server web ui - job counter values for map/reduce not shown properly > > > Key: MAPREDUCE-3686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Bhallamudi Venkata Siva Kamesh >Priority: Critical > Attachments: MAPREDUCE-3686.patch, MAPREDUCE-3686.patch, > jobcounter1.png, jobcounter2.png, jobcounter3.png > > > Looking at the job counters page on the history server for a finished job, it > shows 3 columns for each counter - map, reduce, total. The total appears > correct, but map and reduce columns are always zero. If you click on a > particular column it does show you the map/reduce task and the value of each > one. > Going to attach screenshots shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200505#comment-13200505
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Hdfs-trunk-Commit #1732 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1732/])
MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240590
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200504#comment-13200504
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Common-0.23-Commit #488 (See
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/488/])
MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) - Merging r1240590
from trunk.
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240591
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200503#comment-13200503
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Hdfs-0.23-Commit #478 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/478/])
MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev) - Merging r1240590
from trunk.
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240591
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200501#comment-13200501
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Common-trunk-Commit #1660 (See
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1660/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240587
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/trunk/hadoop-project/src/site/site.xml
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 2:18.427s
> [INFO] Finished at: Thu Feb 02
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200502#comment-13200502
]
Hudson commented on MAPREDUCE-3723:
---
Integrated in Hadoop-Common-trunk-Commit #1660 (See
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1660/])
MAPREDUCE-3723. TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly
asserting tests (Bhallamudi Venkata Siva Kamesh via mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240590
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200500#comment-13200500
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Hdfs-0.23-Commit #477 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/477/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev) -
Merging r1240587 from trunk
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240588
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site/site.xml
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
> --
[jira] [Updated] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3723:
-
Resolution: Fixed
Fix Version/s: 0.23.1
Status: Resolved (was: Patch Available)
I just committed this. Thanks Bhallamudi!
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200498#comment-13200498
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Common-0.23-Commit #487 (See
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/487/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev) -
Merging r1240587 from trunk
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240588
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/branches/branch-0.23/hadoop-project/src/site/site.xml
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
> --
[jira] [Commented] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200497#comment-13200497
]
Hudson commented on MAPREDUCE-3791:
---
Integrated in Hadoop-Hdfs-trunk-Commit #1731 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1731/])
MAPREDUCE-3791. can't build site in hadoop-yarn-server-common. (mahadev)
mahadev :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240587
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
* /hadoop/common/trunk/hadoop-project/src/site/site.xml
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 2:18.427s
> [INFO] Finished at: Thu Feb 02 10:3
[jira] [Commented] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200495#comment-13200495
]
Mahadev konar commented on MAPREDUCE-3723:
--
+1 the patch looks good. Good catch Bhallamudi!
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3723) TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar reassigned MAPREDUCE-3723:
Assignee: Bhallamudi Venkata Siva Kamesh
> TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests
> -
>
> Key: MAPREDUCE-3723
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3723
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test, webapps
>Affects Versions: 0.23.1, 0.24.0
>Reporter: Bhallamudi Venkata Siva Kamesh
>Assignee: Bhallamudi Venkata Siva Kamesh
> Attachments: MAPREDUCE-3723.patch
>
>
> While testing a patch for one of the MR issues, I found TestAMWebServicesJobs
> & TestHSWebServicesJobs incorrectly asserting tests.
> Moreover tests may fail if
> {noformat}
> index of counterGroups > #counters in a particular counterGroup
> {noformat}
> {code:title=TestAMWebServicesJobs.java|borderStyle=solid}
> for (int j = 0; j < counters.length(); j++) {
> JSONObject counter = counters.getJSONObject(i);
> {code}
> where is *i* is index of outer loop. It should be *j* instead of *i*.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3436) jobhistory link may be broken depending on the interface it is listening on
[
https://issues.apache.org/jira/browse/MAPREDUCE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200494#comment-13200494
]
Mahadev konar commented on MAPREDUCE-3436:
--
@Ahmed/Sid,
Any update on this one?
> jobhistory link may be broken depending on the interface it is listening on
> ---
>
> Key: MAPREDUCE-3436
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3436
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, webapps
>Affects Versions: 0.23.0, 0.23.1
>Reporter: Bruno Mahé
>Assignee: Ahmed Radwan
> Labels: bigtop
> Attachments: MAPREDUCE-3436.patch, MAPREDUCE-3436_rev2.patch
>
>
> On the following page : http://:8088/cluster/apps
> There are links to the history for each application. None of them can be
> reached since they all point to the ip 0.0.0.0. For instance:
> http://0.0.0.0:8088/proxy/application_1321658790349_0002/jobhistory/job/job_1321658790349_2_2
> Am I missing something?
> [root@bigtop-fedora-15 ~]# jps
> 9968 ResourceManager
> 1495 NameNode
> 1645 DataNode
> 12935 Jps
> 11140 -- process information unavailable
> 5309 JobHistoryServer
> 10237 NodeManager
> [root@bigtop-fedora-15 ~]# netstat -tlpn | grep 8088
> tcp0 0 :::8088 :::*
> LISTEN 9968/java
> For reference, here is my configuration:
> root@bigtop-fedora-15 ~]# cat /etc/yarn/conf/yarn-site.xml
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce.shuffle
>
>
> yarn.nodemanager.aux-services.mapreduce.shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> mapreduce.admin.user.env
>
> CLASSPATH=/etc/hadoop/conf/*:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/hdfs-site.xml
>
>
>
> dfs.replication
> 1
>
>
> dfs.permissions
> false
>
>
>
> dfs.name.dir
> /var/lib/hadoop/cache/hadoop/dfs/name
>
>
> [root@bigtop-fedora-15 ~]# cat /etc/hadoop/conf/core-site.xml
>
>
>
> fs.default.name
> hdfs://localhost:8020
>
>
> hadoop.tmp.dir
> /var/lib/hadoop/cache/${user.name}
>
>
>
> hadoop.proxyuser.oozie.hosts
> *
>
>
> hadoop.proxyuser.oozie.groups
> *
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3791:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I just committed this!
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 2:18.427s
> [INFO] Finished at: Thu Feb 02 10:31:35 PST 2012
> [INFO] Final Memory: 321M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.6:run (default) on project
> hadoop-yarn-server-common: An Ant BuildException has occured: Warning: Could
> not find file
> /home/rvs/src/apache/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml
> to copy. -> [Help 1]
>
[jira] [Updated] (MAPREDUCE-3791) can't build site in hadoop-yarn-server-common
[
https://issues.apache.org/jira/browse/MAPREDUCE-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3791:
-
Assignee: Mahadev konar (was: Roman Shaposhnik)
Thanks for verifying Roman. Committing this now.
> can't build site in hadoop-yarn-server-common
> -
>
> Key: MAPREDUCE-3791
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3791
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation, mrv2
>Affects Versions: 0.23.1
>Reporter: Roman Shaposhnik
>Assignee: Mahadev konar
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3791.patch
>
>
> Here's how to reproduce:
> {noformat}
> $ mvn site site:stage -DskipTests -DskipTest -DskipITs
>
> main:
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main SUCCESS [49.017s]
> [INFO] Apache Hadoop Project POM . SUCCESS [5.152s]
> [INFO] Apache Hadoop Annotations . SUCCESS [4.973s]
> [INFO] Apache Hadoop Project Dist POM SUCCESS [4.514s]
> [INFO] Apache Hadoop Assemblies .. SUCCESS [4.334s]
> [INFO] Apache Hadoop Auth SUCCESS [5.215s]
> [INFO] Apache Hadoop Auth Examples ... SUCCESS [4.051s]
> [INFO] Apache Hadoop Common .. SUCCESS [4.111s]
> [INFO] Apache Hadoop Common Project .. SUCCESS [4.198s]
> [INFO] Apache Hadoop HDFS SUCCESS [5.373s]
> [INFO] Apache Hadoop HttpFS .. SUCCESS [22.549s]
> [INFO] Apache Hadoop HDFS Project SUCCESS [4.440s]
> [INFO] hadoop-yarn ... SUCCESS [5.250s]
> [INFO] hadoop-yarn-api ... SUCCESS [4.579s]
> [INFO] hadoop-yarn-common SUCCESS [4.268s]
> [INFO] hadoop-yarn-server SUCCESS [4.408s]
> [INFO] hadoop-yarn-server-common . FAILURE [0.035s]
> [INFO] hadoop-yarn-server-nodemanager SKIPPED
> [INFO] hadoop-yarn-server-web-proxy .. SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager SKIPPED
> [INFO] hadoop-yarn-server-tests .. SKIPPED
> [INFO] hadoop-mapreduce-client ... SKIPPED
> [INFO] hadoop-mapreduce-client-core .. SKIPPED
> [INFO] hadoop-yarn-applications .. SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell . SKIPPED
> [INFO] hadoop-yarn-site .. SKIPPED
> [INFO] hadoop-mapreduce-client-common SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED
> [INFO] hadoop-mapreduce-client-app ... SKIPPED
> [INFO] hadoop-mapreduce-client-hs SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient . SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .. SKIPPED
> [INFO] hadoop-mapreduce .. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming . SKIPPED
> [INFO] Apache Hadoop Distributed Copy SKIPPED
> [INFO] Apache Hadoop Archives SKIPPED
> [INFO] Apache Hadoop Rumen ... SKIPPED
> [INFO] Apache Hadoop Extras .. SKIPPED
> [INFO] Apache Hadoop Tools Dist .. SKIPPED
> [INFO] Apache Hadoop Tools ... SKIPPED
> [INFO] Apache Hadoop Distribution SKIPPED
> [INFO] Apache Hadoop Client .. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 2:18.427s
> [INFO] Finished at: Thu Feb 02 10:31:35 PST 2012
> [INFO] Final Memory: 321M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.6:run (default) on project
> hadoop-yarn-server-common: An Ant BuildException has occured: Warning: Could
> not find file
> /home/rvs/src/apache/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml
> to copy. -> [Help 1]
> [ERROR]
> [ERROR] To
[jira] [Updated] (MAPREDUCE-3697) Hadoop Counters API limits Oozie's working across different hadoop versions
[ https://issues.apache.org/jira/browse/MAPREDUCE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3697: - Attachment: MAPREDUCE-3697.patch This should take care of the findbugs and test issue. > Hadoop Counters API limits Oozie's working across different hadoop versions > --- > > Key: MAPREDUCE-3697 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3697 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: John George >Assignee: Mahadev konar >Priority: Blocker > Attachments: MAPREDUCE-3697-facade.patch, > MAPREDUCE-3697-facadeV2.patch, MAPREDUCE-3697-facadeV3.patch, > MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, > MAPREDUCE-3697.patch, MAPREDUCE-3697.patch > > > Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in > hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). > Its > parent class AbstractCounters implements it. This is different from hadoop20X. > As a result, Oozie compiled with either hadoop version does not work with the > other version. > A specific scenario, Oozie compiled with .23 and run against 205, does not > update job status owing to a Counters API exception. > Will explicit re-compilation against the relevant hadoop jars be required each > time? This will prevent launching a uniform Oozie version across different > clusters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3697) Hadoop Counters API limits Oozie's working across different hadoop versions
[ https://issues.apache.org/jira/browse/MAPREDUCE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3697: - Status: Patch Available (was: Open) > Hadoop Counters API limits Oozie's working across different hadoop versions > --- > > Key: MAPREDUCE-3697 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3697 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: John George >Assignee: Mahadev konar >Priority: Blocker > Attachments: MAPREDUCE-3697-facade.patch, > MAPREDUCE-3697-facadeV2.patch, MAPREDUCE-3697-facadeV3.patch, > MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, > MAPREDUCE-3697.patch, MAPREDUCE-3697.patch > > > Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in > hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). > Its > parent class AbstractCounters implements it. This is different from hadoop20X. > As a result, Oozie compiled with either hadoop version does not work with the > other version. > A specific scenario, Oozie compiled with .23 and run against 205, does not > update job status owing to a Counters API exception. > Will explicit re-compilation against the relevant hadoop jars be required each > time? This will prevent launching a uniform Oozie version across different > clusters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3697) Hadoop Counters API limits Oozie's working across different hadoop versions
[ https://issues.apache.org/jira/browse/MAPREDUCE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3697: - Status: Open (was: Patch Available) > Hadoop Counters API limits Oozie's working across different hadoop versions > --- > > Key: MAPREDUCE-3697 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3697 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: John George >Assignee: Mahadev konar >Priority: Blocker > Attachments: MAPREDUCE-3697-facade.patch, > MAPREDUCE-3697-facadeV2.patch, MAPREDUCE-3697-facadeV3.patch, > MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, MAPREDUCE-3697.patch, > MAPREDUCE-3697.patch > > > Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in > hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). > Its > parent class AbstractCounters implements it. This is different from hadoop20X. > As a result, Oozie compiled with either hadoop version does not work with the > other version. > A specific scenario, Oozie compiled with .23 and run against 205, does not > update job status owing to a Counters API exception. > Will explicit re-compilation against the relevant hadoop jars be required each > time? This will prevent launching a uniform Oozie version across different > clusters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3803) HDFS-2864 broke ant compilation
[
https://issues.apache.org/jira/browse/MAPREDUCE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200454#comment-13200454
]
Hudson commented on MAPREDUCE-3803:
---
Integrated in Hadoop-Mapreduce-trunk #979 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/979/])
MAPREDUCE-3803. Fix broken build of raid contrib due to HDFS-2864.
Contributed by Ravi Prakash.
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240441
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
> HDFS-2864 broke ant compilation
> ---
>
> Key: MAPREDUCE-3803
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3803
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: build
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Attachments: HDFS-2888.patch, MAPREDUCE-3803.patch
>
>
> compile:
> [echo] contrib: raid
> [javac]
> /hadoop-mapreduce-project/src/contrib/build-contrib.xml:194:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 28 source files to
> /hadoop-mapreduce-project/build/contrib/raid/classes
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:111:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] if (version != FSDataset.METADATA_VERSION) {
> [javac] ^
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:649:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] mdOut.writeShort(FSDataset.METADATA_VERSION);
> [javac] ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> BUILD FAILED
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3727) jobtoken location property in jobconf refers to wrong jobtoken file
[ https://issues.apache.org/jira/browse/MAPREDUCE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200451#comment-13200451 ] Hudson commented on MAPREDUCE-3727: --- Integrated in Hadoop-Mapreduce-trunk #979 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/979/]) MAPREDUCE-3727. jobtoken location property in jobconf refers to wrong jobtoken file (tucu) tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240410 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java > jobtoken location property in jobconf refers to wrong jobtoken file > --- > > Key: MAPREDUCE-3727 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3727 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.23.1, 1.0.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur >Priority: Critical > Fix For: 0.23.1, 1.1.0 > > Attachments: MAPREDUCE-3727-branch1.patch, MAPREDUCE-3727.patch > > > Oozie launcher job (for MR/Pig/Hive/Sqoop action) reads the location of the > jobtoken file from the *HADOOP_TOKEN_FILE_LOCATION* ENV var and seeds it as > the *mapreduce.job.credentials.binary* property in the jobconf that will be > used to launch the real (MR/Pig/Hive/Sqoop) job. > The MR/Pig/Hive/Sqoop submission code (via Hadoop job submission) uses > correctly the injected *mapreduce.job.credentials.binary* property to load > the credentials and submit their MR jobs. > The problem is that the *mapreduce.job.credentials.binary* property also > makes it to the tasks of the MR/Pig/Hive/Sqoop MR jobs. > If for some reason the MR/Pig/Hive/Sqoop MR code does some logic that > triggers the credential loading, because the property is set, the credential > loading fails trying to load a jobtoken file of the launcher job which does > not exists in the context of the MR/Pig/Hive/Sqoop jobs. > More specifically, we are seeing this happening with certain hive queries > that trigger a conditional code within their RowContainer which then uses the > FileInputFormat.getSplits() and then the TokenCache tries to load credentials > for a file that is for the wrong job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3760) Blacklisted NMs should not appear in Active nodes list
[ https://issues.apache.org/jira/browse/MAPREDUCE-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200453#comment-13200453 ] Hudson commented on MAPREDUCE-3760: --- Integrated in Hadoop-Mapreduce-trunk #979 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/979/]) MAPREDUCE-3760. Changed active nodes list to not contain unhealthy nodes on the webUI and metrics. (vinodkv) vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240421 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java > Blacklisted NMs should not appear in Active nodes list > -- > > Key: MAPREDUCE-3760 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3760 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3760-20120130.txt, > MAPREDUCE-3760-20120201.txt, MAPREDUCE-3760-20120202.txt > > > Blacklisted NMs appear in both "Active Nodes" and "Unhealthy nodes" on the RM > UI. This should be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3711) AppMaster recovery for Medium to large jobs take long time
[ https://issues.apache.org/jira/browse/MAPREDUCE-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200447#comment-13200447 ] Hudson commented on MAPREDUCE-3711: --- Integrated in Hadoop-Mapreduce-trunk #979 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/979/]) MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. Contributed by Robert Joseph Evans. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240413 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/recover/RecoveryService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTaskCommit.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java > AppMaster recovery for Medium to large jobs take long time > -- > > Key: MAPREDUCE-3711 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3711 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Siddharth Seth >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3711-20120203.txt, MR-3711.txt, MR-3711.txt, > MR-3711.txt > > > Reported by [~karams] > yarn.resourcemanager.am.max-retries=2 > Ran test cases with sort job on 350 scale having 16800 maps and 680 reduces -: > 1. After 70 secs of Job Sumbission Am is killed using kill -9, around 3900 > maps were completed and 680 reduces were > scheduled, Second AM got restart. Job got completed in 980 secs. AM took very > less time to recover. > 2. After 150 secs of Job Sumbission AM is killed using kill -9, around 90% > maps were completed and 680 reduces were > scheduled , Second AM got restart Job got completed in 1000 secs. AM got > revocer. > 3. After 150 secs of Job Sumbission AM as killed using kill -9, almost all > maps were completed and only 680 reduces > were running, Recovery was too slow, AM was still revocering after 1hr :40 > mis when I killed the run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working app master and job history UI's
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200445#comment-13200445 ] Hudson commented on MAPREDUCE-3417: --- Integrated in Hadoop-Mapreduce-trunk #979 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/979/]) MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240428 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java > job access controls not working app master and job history UI's > --- > > Key: MAPREDUCE-3417 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch > > > tested with security on, no filters defined for httpserver, job acls set so > that only I could view/modify the job. Then went to the web ui to app master > and job history server and both allowed me to view the job details. The > webui shows the user "webuser". The RM properly rejected my request > although it was using user "Dr.Who". > The exception shown in the log is: > 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser > 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception > trying to get groups for user webuser > org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user > at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) > at org.apache.hadoop.util.Shell.run(Shell.java:188) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:467) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:450) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55) > at org.apache.hadoop.security.Groups.getGroups(Groups.java:88) > at > org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043) > at > org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221) > at > org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103) > at > org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3803) HDFS-2864 broke ant compilation
[
https://issues.apache.org/jira/browse/MAPREDUCE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200431#comment-13200431
]
Hudson commented on MAPREDUCE-3803:
---
Integrated in Hadoop-Mapreduce-0.23-Build #181 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/181/])
MAPREDUCE-3803. Merge change r1240441 from trunk
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240444
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
> HDFS-2864 broke ant compilation
> ---
>
> Key: MAPREDUCE-3803
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3803
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: build
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Attachments: HDFS-2888.patch, MAPREDUCE-3803.patch
>
>
> compile:
> [echo] contrib: raid
> [javac]
> /hadoop-mapreduce-project/src/contrib/build-contrib.xml:194:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 28 source files to
> /hadoop-mapreduce-project/build/contrib/raid/classes
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:111:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] if (version != FSDataset.METADATA_VERSION) {
> [javac] ^
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:649:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] mdOut.writeShort(FSDataset.METADATA_VERSION);
> [javac] ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> BUILD FAILED
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3727) jobtoken location property in jobconf refers to wrong jobtoken file
[ https://issues.apache.org/jira/browse/MAPREDUCE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200428#comment-13200428 ] Hudson commented on MAPREDUCE-3727: --- Integrated in Hadoop-Mapreduce-0.23-Build #181 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/181/]) Merge -r 1240409:1240410 from trunk to branch. FIXES: MAPREDUCE-3727 tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240411 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java > jobtoken location property in jobconf refers to wrong jobtoken file > --- > > Key: MAPREDUCE-3727 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3727 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.23.1, 1.0.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur >Priority: Critical > Fix For: 0.23.1, 1.1.0 > > Attachments: MAPREDUCE-3727-branch1.patch, MAPREDUCE-3727.patch > > > Oozie launcher job (for MR/Pig/Hive/Sqoop action) reads the location of the > jobtoken file from the *HADOOP_TOKEN_FILE_LOCATION* ENV var and seeds it as > the *mapreduce.job.credentials.binary* property in the jobconf that will be > used to launch the real (MR/Pig/Hive/Sqoop) job. > The MR/Pig/Hive/Sqoop submission code (via Hadoop job submission) uses > correctly the injected *mapreduce.job.credentials.binary* property to load > the credentials and submit their MR jobs. > The problem is that the *mapreduce.job.credentials.binary* property also > makes it to the tasks of the MR/Pig/Hive/Sqoop MR jobs. > If for some reason the MR/Pig/Hive/Sqoop MR code does some logic that > triggers the credential loading, because the property is set, the credential > loading fails trying to load a jobtoken file of the launcher job which does > not exists in the context of the MR/Pig/Hive/Sqoop jobs. > More specifically, we are seeing this happening with certain hive queries > that trigger a conditional code within their RowContainer which then uses the > FileInputFormat.getSplits() and then the TokenCache tries to load credentials > for a file that is for the wrong job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3760) Blacklisted NMs should not appear in Active nodes list
[ https://issues.apache.org/jira/browse/MAPREDUCE-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200430#comment-13200430 ] Hudson commented on MAPREDUCE-3760: --- Integrated in Hadoop-Mapreduce-0.23-Build #181 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/181/]) MAPREDUCE-3760. Changed active nodes list to not contain unhealthy nodes on the webUI and metrics. (vinodkv) svn merge --ignore-ancestry -c 1240421 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240423 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java > Blacklisted NMs should not appear in Active nodes list > -- > > Key: MAPREDUCE-3760 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3760 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3760-20120130.txt, > MAPREDUCE-3760-20120201.txt, MAPREDUCE-3760-20120202.txt > > > Blacklisted NMs appear in both "Active Nodes" and "Unhealthy nodes" on the RM > UI. This should be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3711) AppMaster recovery for Medium to large jobs take long time
[ https://issues.apache.org/jira/browse/MAPREDUCE-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200424#comment-13200424 ] Hudson commented on MAPREDUCE-3711: --- Integrated in Hadoop-Mapreduce-0.23-Build #181 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/181/]) MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. Contributed by Robert Joseph Evans. svn merge --ignore-ancestry -c 1240413 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240414 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/recover/RecoveryService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTaskCommit.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java > AppMaster recovery for Medium to large jobs take long time > -- > > Key: MAPREDUCE-3711 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3711 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Siddharth Seth >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3711-20120203.txt, MR-3711.txt, MR-3711.txt, > MR-3711.txt > > > Reported by [~karams] > yarn.resourcemanager.am.max-retries=2 > Ran test cases with sort job on 350 scale having 16800 maps and 680 reduces -: > 1. After 70 secs of Job Sumbission Am is killed using kill -9, around 3900 > maps were completed and 680 reduces were > scheduled, Second AM got restart. Job got completed in 980 secs. AM took very > less time to recover. > 2. After 150 secs of Job Sumbission AM is killed using kill -9, around 90% > maps were completed and 680 reduces were > scheduled , Second AM got restart Job got completed in 1000 secs. AM got > revocer. > 3. After 150 secs of Job Sumbission AM as killed using kill -9, almost all > maps were completed and only 680 reduces > were running, Recovery was too slow, AM was still revocering after 1hr :40 > mis when I killed the run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: ht
[jira] [Commented] (MAPREDUCE-3417) job access controls not working app master and job history UI's
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200423#comment-13200423 ] Hudson commented on MAPREDUCE-3417: --- Integrated in Hadoop-Mapreduce-0.23-Build #181 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/181/]) MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles. svn merge --ignore-ancestry -c 1240428 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240429 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java > job access controls not working app master and job history UI's > --- > > Key: MAPREDUCE-3417 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch > > > tested with security on, no filters defined for httpserver, job acls set so > that only I could view/modify the job. Then went to the web ui to app master > and job history server and both allowed me to view the job details. The > webui shows the user "webuser". The RM properly rejected my request > although it was using user "Dr.Who". > The exception shown in the log is: > 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser > 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception > trying to get groups for user webuser > org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user > at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) > at org.apache.hadoop.util.Shell.run(Shell.java:188) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:467) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:450) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55) > at org.apache.hadoop.security.Groups.getGroups(Groups.java:88) > at > org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043) > at > org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221) > at > org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103) > at > org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3803) HDFS-2864 broke ant compilation
[
https://issues.apache.org/jira/browse/MAPREDUCE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200411#comment-13200411
]
Hudson commented on MAPREDUCE-3803:
---
Integrated in Hadoop-Hdfs-0.23-Build #159 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/159/])
MAPREDUCE-3803. Merge change r1240441 from trunk
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240444
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
> HDFS-2864 broke ant compilation
> ---
>
> Key: MAPREDUCE-3803
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3803
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: build
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Attachments: HDFS-2888.patch, MAPREDUCE-3803.patch
>
>
> compile:
> [echo] contrib: raid
> [javac]
> /hadoop-mapreduce-project/src/contrib/build-contrib.xml:194:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 28 source files to
> /hadoop-mapreduce-project/build/contrib/raid/classes
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:111:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] if (version != FSDataset.METADATA_VERSION) {
> [javac] ^
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:649:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] mdOut.writeShort(FSDataset.METADATA_VERSION);
> [javac] ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> BUILD FAILED
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3727) jobtoken location property in jobconf refers to wrong jobtoken file
[ https://issues.apache.org/jira/browse/MAPREDUCE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200408#comment-13200408 ] Hudson commented on MAPREDUCE-3727: --- Integrated in Hadoop-Hdfs-0.23-Build #159 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/159/]) Merge -r 1240409:1240410 from trunk to branch. FIXES: MAPREDUCE-3727 tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240411 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java > jobtoken location property in jobconf refers to wrong jobtoken file > --- > > Key: MAPREDUCE-3727 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3727 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.23.1, 1.0.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur >Priority: Critical > Fix For: 0.23.1, 1.1.0 > > Attachments: MAPREDUCE-3727-branch1.patch, MAPREDUCE-3727.patch > > > Oozie launcher job (for MR/Pig/Hive/Sqoop action) reads the location of the > jobtoken file from the *HADOOP_TOKEN_FILE_LOCATION* ENV var and seeds it as > the *mapreduce.job.credentials.binary* property in the jobconf that will be > used to launch the real (MR/Pig/Hive/Sqoop) job. > The MR/Pig/Hive/Sqoop submission code (via Hadoop job submission) uses > correctly the injected *mapreduce.job.credentials.binary* property to load > the credentials and submit their MR jobs. > The problem is that the *mapreduce.job.credentials.binary* property also > makes it to the tasks of the MR/Pig/Hive/Sqoop MR jobs. > If for some reason the MR/Pig/Hive/Sqoop MR code does some logic that > triggers the credential loading, because the property is set, the credential > loading fails trying to load a jobtoken file of the launcher job which does > not exists in the context of the MR/Pig/Hive/Sqoop jobs. > More specifically, we are seeing this happening with certain hive queries > that trigger a conditional code within their RowContainer which then uses the > FileInputFormat.getSplits() and then the TokenCache tries to load credentials > for a file that is for the wrong job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3760) Blacklisted NMs should not appear in Active nodes list
[ https://issues.apache.org/jira/browse/MAPREDUCE-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200410#comment-13200410 ] Hudson commented on MAPREDUCE-3760: --- Integrated in Hadoop-Hdfs-0.23-Build #159 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/159/]) MAPREDUCE-3760. Changed active nodes list to not contain unhealthy nodes on the webUI and metrics. (vinodkv) svn merge --ignore-ancestry -c 1240421 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240423 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java > Blacklisted NMs should not appear in Active nodes list > -- > > Key: MAPREDUCE-3760 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3760 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3760-20120130.txt, > MAPREDUCE-3760-20120201.txt, MAPREDUCE-3760-20120202.txt > > > Blacklisted NMs appear in both "Active Nodes" and "Unhealthy nodes" on the RM > UI. This should be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3711) AppMaster recovery for Medium to large jobs take long time
[ https://issues.apache.org/jira/browse/MAPREDUCE-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200404#comment-13200404 ] Hudson commented on MAPREDUCE-3711: --- Integrated in Hadoop-Hdfs-0.23-Build #159 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/159/]) MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. Contributed by Robert Joseph Evans. svn merge --ignore-ancestry -c 1240413 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240414 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/recover/RecoveryService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTaskCommit.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java > AppMaster recovery for Medium to large jobs take long time > -- > > Key: MAPREDUCE-3711 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3711 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Siddharth Seth >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3711-20120203.txt, MR-3711.txt, MR-3711.txt, > MR-3711.txt > > > Reported by [~karams] > yarn.resourcemanager.am.max-retries=2 > Ran test cases with sort job on 350 scale having 16800 maps and 680 reduces -: > 1. After 70 secs of Job Sumbission Am is killed using kill -9, around 3900 > maps were completed and 680 reduces were > scheduled, Second AM got restart. Job got completed in 980 secs. AM took very > less time to recover. > 2. After 150 secs of Job Sumbission AM is killed using kill -9, around 90% > maps were completed and 680 reduces were > scheduled , Second AM got restart Job got completed in 1000 secs. AM got > revocer. > 3. After 150 secs of Job Sumbission AM as killed using kill -9, almost all > maps were completed and only 680 reduces > were running, Recovery was too slow, AM was still revocering after 1hr :40 > mis when I killed the run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issu
[jira] [Commented] (MAPREDUCE-3417) job access controls not working app master and job history UI's
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200403#comment-13200403 ] Hudson commented on MAPREDUCE-3417: --- Integrated in Hadoop-Hdfs-0.23-Build #159 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/159/]) MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles. svn merge --ignore-ancestry -c 1240428 ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240429 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java > job access controls not working app master and job history UI's > --- > > Key: MAPREDUCE-3417 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch > > > tested with security on, no filters defined for httpserver, job acls set so > that only I could view/modify the job. Then went to the web ui to app master > and job history server and both allowed me to view the job details. The > webui shows the user "webuser". The RM properly rejected my request > although it was using user "Dr.Who". > The exception shown in the log is: > 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser > 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception > trying to get groups for user webuser > org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user > at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) > at org.apache.hadoop.util.Shell.run(Shell.java:188) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:467) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:450) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55) > at org.apache.hadoop.security.Groups.getGroups(Groups.java:88) > at > org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043) > at > org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221) > at > org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103) > at > org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3727) jobtoken location property in jobconf refers to wrong jobtoken file
[ https://issues.apache.org/jira/browse/MAPREDUCE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200397#comment-13200397 ] Hudson commented on MAPREDUCE-3727: --- Integrated in Hadoop-Hdfs-trunk #946 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/946/]) MAPREDUCE-3727. jobtoken location property in jobconf refers to wrong jobtoken file (tucu) tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240410 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java > jobtoken location property in jobconf refers to wrong jobtoken file > --- > > Key: MAPREDUCE-3727 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3727 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.23.1, 1.0.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur >Priority: Critical > Fix For: 0.23.1, 1.1.0 > > Attachments: MAPREDUCE-3727-branch1.patch, MAPREDUCE-3727.patch > > > Oozie launcher job (for MR/Pig/Hive/Sqoop action) reads the location of the > jobtoken file from the *HADOOP_TOKEN_FILE_LOCATION* ENV var and seeds it as > the *mapreduce.job.credentials.binary* property in the jobconf that will be > used to launch the real (MR/Pig/Hive/Sqoop) job. > The MR/Pig/Hive/Sqoop submission code (via Hadoop job submission) uses > correctly the injected *mapreduce.job.credentials.binary* property to load > the credentials and submit their MR jobs. > The problem is that the *mapreduce.job.credentials.binary* property also > makes it to the tasks of the MR/Pig/Hive/Sqoop MR jobs. > If for some reason the MR/Pig/Hive/Sqoop MR code does some logic that > triggers the credential loading, because the property is set, the credential > loading fails trying to load a jobtoken file of the launcher job which does > not exists in the context of the MR/Pig/Hive/Sqoop jobs. > More specifically, we are seeing this happening with certain hive queries > that trigger a conditional code within their RowContainer which then uses the > FileInputFormat.getSplits() and then the TokenCache tries to load credentials > for a file that is for the wrong job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3760) Blacklisted NMs should not appear in Active nodes list
[ https://issues.apache.org/jira/browse/MAPREDUCE-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200399#comment-13200399 ] Hudson commented on MAPREDUCE-3760: --- Integrated in Hadoop-Hdfs-trunk #946 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/946/]) MAPREDUCE-3760. Changed active nodes list to not contain unhealthy nodes on the webUI and metrics. (vinodkv) vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240421 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java > Blacklisted NMs should not appear in Active nodes list > -- > > Key: MAPREDUCE-3760 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3760 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.1 >Reporter: Ramya Sunil >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3760-20120130.txt, > MAPREDUCE-3760-20120201.txt, MAPREDUCE-3760-20120202.txt > > > Blacklisted NMs appear in both "Active Nodes" and "Unhealthy nodes" on the RM > UI. This should be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3803) HDFS-2864 broke ant compilation
[
https://issues.apache.org/jira/browse/MAPREDUCE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200400#comment-13200400
]
Hudson commented on MAPREDUCE-3803:
---
Integrated in Hadoop-Hdfs-trunk #946 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/946/])
MAPREDUCE-3803. Fix broken build of raid contrib due to HDFS-2864.
Contributed by Ravi Prakash.
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240441
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
> HDFS-2864 broke ant compilation
> ---
>
> Key: MAPREDUCE-3803
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3803
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: build
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Attachments: HDFS-2888.patch, MAPREDUCE-3803.patch
>
>
> compile:
> [echo] contrib: raid
> [javac]
> /hadoop-mapreduce-project/src/contrib/build-contrib.xml:194:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 28 source files to
> /hadoop-mapreduce-project/build/contrib/raid/classes
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:111:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] if (version != FSDataset.METADATA_VERSION) {
> [javac] ^
> [javac]
> /hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:649:
> cannot find symbol
> [javac] symbol : variable METADATA_VERSION
> [javac] location: class org.apache.hadoop.hdfs.server.datanode.FSDataset
> [javac] mdOut.writeShort(FSDataset.METADATA_VERSION);
> [javac] ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> BUILD FAILED
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3711) AppMaster recovery for Medium to large jobs take long time
[ https://issues.apache.org/jira/browse/MAPREDUCE-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200393#comment-13200393 ] Hudson commented on MAPREDUCE-3711: --- Integrated in Hadoop-Hdfs-trunk #946 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/946/]) MAPREDUCE-3711. Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. Contributed by Robert Joseph Evans. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240413 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/recover/RecoveryService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTaskCommit.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java > AppMaster recovery for Medium to large jobs take long time > -- > > Key: MAPREDUCE-3711 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3711 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Siddharth Seth >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3711-20120203.txt, MR-3711.txt, MR-3711.txt, > MR-3711.txt > > > Reported by [~karams] > yarn.resourcemanager.am.max-retries=2 > Ran test cases with sort job on 350 scale having 16800 maps and 680 reduces -: > 1. After 70 secs of Job Sumbission Am is killed using kill -9, around 3900 > maps were completed and 680 reduces were > scheduled, Second AM got restart. Job got completed in 980 secs. AM took very > less time to recover. > 2. After 150 secs of Job Sumbission AM is killed using kill -9, around 90% > maps were completed and 680 reduces were > scheduled , Second AM got restart Job got completed in 1000 secs. AM got > revocer. > 3. After 150 secs of Job Sumbission AM as killed using kill -9, almost all > maps were completed and only 680 reduces > were running, Recovery was too slow, AM was still revocering after 1hr :40 > mis when I killed the run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working app master and job history UI's
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200391#comment-13200391 ] Hudson commented on MAPREDUCE-3417: --- Integrated in Hadoop-Hdfs-trunk #946 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/946/]) MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240428 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java > job access controls not working app master and job history UI's > --- > > Key: MAPREDUCE-3417 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, > MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch > > > tested with security on, no filters defined for httpserver, job acls set so > that only I could view/modify the job. Then went to the web ui to app master > and job history server and both allowed me to view the job details. The > webui shows the user "webuser". The RM properly rejected my request > although it was using user "Dr.Who". > The exception shown in the log is: > 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser > 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception > trying to get groups for user webuser > org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user > at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) > at org.apache.hadoop.util.Shell.run(Shell.java:188) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:467) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:450) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86) > at > org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55) > at org.apache.hadoop.security.Groups.getGroups(Groups.java:88) > at > org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043) > at > org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221) > at > org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103) > at > org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313) > at > org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
