[jira] [Commented] (MAPREDUCE-3314) Support AM specific blacklisting of nodes on the RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139942#comment-13139942 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3314: Thanks for filing this. I ran into the same issue but was lazy enough not to create it. Essentially what happens for me is that on a small enough cluster (single node actually), if all nodes are blacklisted by the AM, it will just keep getting containers and wasting them. Support AM specific blacklisting of nodes on the RM --- Key: MAPREDUCE-3314 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3314 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.1 The AM blacklists certain nodes due to task failures but the RM does not know of this and can allocate blacklisted nodes to the AM again and again. RM should support a protocol that allows AM to inform the RM of nodes that it should not allocate to it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139944#comment-13139944 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Common-trunk-Commit #1201 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1201/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195340 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139943#comment-13139943 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Hdfs-0.23-Commit #108 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/108/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. svn merge -c r1195340 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195341 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139945#comment-13139945 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Hdfs-trunk-Commit #1277 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1277/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195340 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139946#comment-13139946 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Common-0.23-Commit #108 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/108/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. svn merge -c r1195340 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195341 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3297) Move Log Related components from yarn-server-nodemanager to yarn-common
[ https://issues.apache.org/jira/browse/MAPREDUCE-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139947#comment-13139947 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3297: Sid, do you want to take a crack at this? After MAPREDUCE-3146, client-core also depends on nodemanager, very ugly. My bet is it is going to be further unwieldly if we don't fix this now. Move Log Related components from yarn-server-nodemanager to yarn-common --- Key: MAPREDUCE-3297 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3297 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0 Reporter: Siddharth Seth or to a separate module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
[ https://issues.apache.org/jira/browse/MAPREDUCE-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139951#comment-13139951 ] Hadoop QA commented on MAPREDUCE-3146: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501571/MAPREDUCE-3146-20111031.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1214//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1214//console This message is automatically generated. Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139952#comment-13139952 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1227 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1227/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195340 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Component/s: nodemanager Description: Container logs aren't getting cleaned up when log-aggregation is disabled. (was: Container logs aren't getting cleaned up causing NMs to fill up.) Priority: Major (was: Critical) If log-aggregation is enabled, the container logs are getting deleted. Changing the title to reflect this new reality. Container logs aren't getting cleaned up Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3241) (Rumen)TraceBuilder throws IllegalArgumentException
[
https://issues.apache.org/jira/browse/MAPREDUCE-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139954#comment-13139954
]
Siddharth Seth commented on MAPREDUCE-3241:
---
Tried the patch - TraceBuilder works with it applied. Looks good, with a follow
up jira to handle the AMStarted event.
(Rumen)TraceBuilder throws IllegalArgumentException
---
Key: MAPREDUCE-3241
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3241
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: mapreduce-3241-v1.1.patch
When we run the TraceBuilder, we get this exception. Output of the
TraceBuilder doesn't contain the map and reduce task information.
{code}
2011-10-21 22:07:17,268 WARN rumen.TraceBuilder (TraceBuilder.java:run(272))
- TraceBuilder got an error while processing the [possibly virtual] file
job_1319214405771_0002-1319214846458-root-word+count-1319214871038-1-1-SUCCEEDED.jhist
within Path
hdfs://10.18.52.57:9000/user/root/null/history/done_intermediate/root/job_1319214405771_0002-1319214846458-root-word+count-1319214871038-1-1-SUCCEEDED.jhist
java.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown
event type
at
org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilder.java:165)
at
org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:304)
at
org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:258)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at
org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
[ https://issues.apache.org/jira/browse/MAPREDUCE-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139956#comment-13139956 ] Siddharth Seth commented on MAPREDUCE-3146: --- Tested on a single node non-secure cluster. Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3103) Implement Job ACLs for MRAppMaster
[ https://issues.apache.org/jira/browse/MAPREDUCE-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3103: - Attachment: MAPREDUCE-3103.patch Looking at the current authentication, the job acl checks in MRAppMaster are no longer necessary since the clients go through acl checks with the RM to get the clientToken. This patch adds, acl checks in the MRApp Web UI and history server webui. The history server RPC already does the necessary acl checks. We need to test the patch on a secure cluster with some web authentication to verify the acl checks in the Web UI's. Implement Job ACLs for MRAppMaster -- Key: MAPREDUCE-3103 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3103 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, security Reporter: Vinod Kumar Vavilapalli Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-3103.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139960#comment-13139960 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Mapreduce-0.23-Commit #117 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/117/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. svn merge -c r1195340 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195341 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
[ https://issues.apache.org/jira/browse/MAPREDUCE-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3146: --- Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this to trunk and branch-0.23. Thanks Sid! Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/main/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2742) [MR-279] [Security] All tokens in YARN + MR should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2742: --- Fix Version/s: (was: 0.24.0) 0.23.1 Marking it for 0.23.1, can't make it for 0.23. [MR-279] [Security] All tokens in YARN + MR should have an expiry interval --- Key: MAPREDUCE-2742 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2742 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, security Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Priority: Blocker Fix For: 0.23.1 Right now none of the tokens and secret-keys have an expiry interval. This needs to be fixed. This ticket should also handle how to renew the tickets when necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/src/main/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Status: Open (was: Patch Available) Container logs aren't getting cleaned up Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Attachment: MR2696_v1.txt MAPREDUCE-3146 is in. Submitting the same patch to Jenkins while I review it. Container logs aren't getting cleaned up Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Status: Patch Available (was: Open) Container logs aren't getting cleaned up Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139985#comment-13139985 ] Hadoop QA commented on MAPREDUCE-2696: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501579/MR2696_v1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1215//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1215//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1215//console This message is automatically generated. Container logs aren't getting cleaned up Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Status: Open (was: Patch Available) Container logs aren't getting cleaned up Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3289) Make use of fadvise in the NM's shuffle handler
[ https://issues.apache.org/jira/browse/MAPREDUCE-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3289: - Status: Open (was: Patch Available) I'm not super convinced we want to play God with the buffer-cache, but it's too late to sermonize now... Minor nits for 23.0: # Please make the default to false # Move the config key/default-value to YarnConfiguration, we don't spread them around now. Make use of fadvise in the NM's shuffle handler --- Key: MAPREDUCE-3289 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3289 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2, nodemanager, performance Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: mr-3289.txt Using the new NativeIO fadvise functions, we can make the NodeManager prefetch map output before it's send over the socket, and drop it out of the fs cache once it's been sent (since it's very rare for an output to have to be re-sent). This improves IO efficiency and reduces cache pollution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Attachment: MAPREDUCE-2696-20111031.txt Modifying the patch and uploading it myself with minor edits. Avoids another review cycle, so: - Renamed -- log-aggregation.enable configuration key to log-aggregation-enable -- log.delete.thread-count configuration key to log.deletion-threads-count -- log.retain.seconds configuration to log.retain-seconds -- APPLICATION_LOG_AGGREAGATION_FINISHED event to APPLICATION_LOG_HANDLING_FINISHED -- LogHnadlerEvent to LogHandlerEvent ( a typo) - Fixed FindBugs warnings. - Uncommented the commented out test in TestNonAggregatingLogHandler, and fixed a bug in that. Container logs aren't getting cleaned up Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Hadoop Flags: Reviewed Summary: Container logs aren't getting cleaned up when LogAggregation is disabled (was: Container logs aren't getting cleaned up) Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Status: Patch Available (was: Open) Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139994#comment-13139994 ] Siddharth Seth commented on MAPREDUCE-3262: --- A specific Transition (ContainerTransition()) isn't really needed for these transitions. Extending what the patch is doing, there's a couple more transitions which could be added and ignored. RESOURCE_FAILED at LOCALIZATION_FAILED RESOURCE_FAILED at KILLING Otherwise, transitions + unit test look good. Handle missing events in failure scenarios -- Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140003#comment-13140003 ] Hadoop QA commented on MAPREDUCE-2696: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501581/MAPREDUCE-2696-20111031.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1216//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1216//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1216//console This message is automatically generated. Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Status: Patch Available (was: Open) Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Status: Open (was: Patch Available) Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Attachment: MAPREDUCE-2696-20111031.1.txt Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3220) ant test TestCombineOutputCollector failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3220: - Status: Patch Available (was: Open) ant test TestCombineOutputCollector failing on trunk Key: MAPREDUCE-3220 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3220 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MAPREDUCE-3220.patch Testsuite: org.apache.hadoop.mapred.TestCombineOutputCollector Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 1.591 sec Testcase: testCustomCollect took 0.363 sec FAILED taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) Testcase: testDefaultCollect took 1.211 sec FAILED taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3220) ant test TestCombineOutputCollector failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3220: - Attachment: MAPREDUCE-3220.patch ant test TestCombineOutputCollector failing on trunk Key: MAPREDUCE-3220 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3220 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MAPREDUCE-3220.patch Testsuite: org.apache.hadoop.mapred.TestCombineOutputCollector Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 1.591 sec Testcase: testCustomCollect took 0.363 sec FAILED taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) Testcase: testDefaultCollect took 1.211 sec FAILED taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3221) ant test TestSubmitJob failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K reassigned MAPREDUCE-3221: Assignee: Devaraj K ant test TestSubmitJob failing on trunk --- Key: MAPREDUCE-3221 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3221 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Devaraj K Priority: Minor Fix For: 0.23.1 Testcase: testJobWithInvalidMemoryReqs took 2.588 sec Testcase: testSecureJobExecution took 4.089 sec FAILED java.io.IOException: org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 69, server = 70) at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:617) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) junit.framework.AssertionFailedError: java.io.IOException: org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 69, server = 70) at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:617) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) at org.apache.hadoop.mapred.TestSubmitJob.testSecureJobExecution(TestSubmitJob.java:270) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3222) ant test TestTaskContext failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K reassigned MAPREDUCE-3222: Assignee: Devaraj K ant test TestTaskContext failing on trunk - Key: MAPREDUCE-3222 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3222 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Devaraj K Priority: Minor Fix For: 0.23.1 Testcase: testContextStatus took 29.977 sec FAILED null expected:map[ sort] but was:map[] junit.framework.ComparisonFailure: null expected:map[ sort] but was:map[] at org.apache.hadoop.mapreduce.TestTaskContext.testContextStatus(TestTaskContext.java:120) Testcase: testMapContextProgress took 17.371 sec Testcase: testReduceContextProgress took 16.267 sec -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3216) ant test TestNoDefaultsJobConf fails on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K reassigned MAPREDUCE-3216: Assignee: Devaraj K ant test TestNoDefaultsJobConf fails on trunk - Key: MAPREDUCE-3216 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3216 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Devaraj K Priority: Minor Fix For: 0.23.1 Testcase: testNoDefaults took 4.703 sec Caused an ERROR Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:118) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:81) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:74) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:460) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:439) at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:809) at org.apache.hadoop.conf.TestNoDefaultsJobConf.testNoDefaults(TestNoDefaultsJobConf.java:83) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3207) TestMRCLI failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K reassigned MAPREDUCE-3207: Assignee: Devaraj K TestMRCLI failing on trunk Key: MAPREDUCE-3207 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3207 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Devaraj K Priority: Minor Fix For: 0.23.1 Attachments: TEST-org.apache.hadoop.cli.TestMRCLI.txt Failing tests: 7: Archive: Deleting a file in archive 8: Archive: Renaming a file in archive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3215) org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K reassigned MAPREDUCE-3215: Assignee: Devaraj K org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk -- Key: MAPREDUCE-3215 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3215 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Devaraj K Priority: Minor Fix For: 0.24.0 Testcase: testNoJobSetupCleanup took 13.271 sec FAILED Number of part-files is 0 and not 1 junit.framework.AssertionFailedError: Number of part-files is 0 and not 1 at org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.submitAndValidateJob(TestNoJobSetupCleanup.java:60) at org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.testNoJobSetupCleanup(TestNoJobSetupCleanup.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3217) ant test TestAuditLogger fails on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K reassigned MAPREDUCE-3217: Assignee: Devaraj K ant test TestAuditLogger fails on trunk --- Key: MAPREDUCE-3217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3217 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Devaraj K Priority: Minor Fix For: 0.23.1 Testcase: testKeyValLogFormat took 0.096 sec Testcase: testAuditLoggerWithoutIP took 0.005 sec Testcase: testAuditLoggerWithIP took 0.417 sec Caused an ERROR java.io.IOException: Unknown protocol: org.apache.hadoop.ipc.TestRPC$TestProtocol at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:615) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) java.io.IOException: java.io.IOException: Unknown protocol: org.apache.hadoop.ipc.TestRPC$TestProtocol at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:615) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) at org.apache.hadoop.ipc.Client.call(Client.java:1085) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:244) at $Proxy6.ping(Unknown Source) at org.apache.hadoop.mapred.TestAuditLogger.testAuditLoggerWithIP(TestAuditLogger.java:150) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140022#comment-13140022 ] Hadoop QA commented on MAPREDUCE-2696: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501586/MAPREDUCE-2696-20111031.1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1217//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1217//console This message is automatically generated. Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-3262: -- Status: Open (was: Patch Available) Handle missing events in failure scenarios -- Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-3262: -- Attachment: MR3262.txt Updated with a couple more transitions and a fix to the unit test... to get this in for 23.0. Handle missing events in failure scenarios -- Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated MAPREDUCE-3262: -- Status: Patch Available (was: Open) Handle missing events in failure scenarios -- Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3220) ant test TestCombineOutputCollector failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140036#comment-13140036 ] Hadoop QA commented on MAPREDUCE-3220: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501589/MAPREDUCE-3220.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1218//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1218//console This message is automatically generated. ant test TestCombineOutputCollector failing on trunk Key: MAPREDUCE-3220 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3220 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.1 Attachments: MAPREDUCE-3220.patch Testsuite: org.apache.hadoop.mapred.TestCombineOutputCollector Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 1.591 sec Testcase: testCustomCollect took 0.363 sec FAILED taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) Testcase: testDefaultCollect took 1.211 sec FAILED taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppStartedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppStartedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140039#comment-13140039 ] Hadoop QA commented on MAPREDUCE-3262: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501591/MR3262.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1219//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1219//console This message is automatically generated. Handle missing events in failure scenarios -- Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2696: --- Resolution: Fixed Status: Resolved (was: Patch Available) I committed this to trunk and branch-0.23. Thanks Sid! Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3262: --- Component/s: nodemanager Summary: A few events are not handled by the NodeManager in failure scenarios (was: Handle missing events in failure scenarios) A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3315) Master-Worker Application on YARN
Master-Worker Application on YARN - Key: MAPREDUCE-3315 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3315 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Sharad Agarwal Assignee: Sharad Agarwal Fix For: 0.24.0 Currently master worker scenarios are forced fit into Map-Reduce. Now with YARN, these can be first class and would benefit real/near realtime workloads and be more effective in using the cluster resources. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
/LogHandlerAppStartedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppStartedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3315) Master-Worker Application on YARN
[ https://issues.apache.org/jira/browse/MAPREDUCE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140059#comment-13140059 ] Sharad Agarwal commented on MAPREDUCE-3315: --- some thoughts: - AM decides the Tasks. - worker keep polling AM for getting the next Task instance. - should be able to dynamically increase or shutdown workers - have minWorkers, maxWorkers, keepAlive Master-Worker Application on YARN - Key: MAPREDUCE-3315 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3315 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Sharad Agarwal Assignee: Sharad Agarwal Fix For: 0.24.0 Currently master worker scenarios are forced fit into Map-Reduce. Now with YARN, these can be first class and would benefit real/near realtime workloads and be more effective in using the cluster resources. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3231) Improve Application Master And Job History UI Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140060#comment-13140060 ] Luke Lu commented on MAPREDUCE-3231: bq. so I don't think it is that big of a deal. Bobby, you're seriously underestimate the potential of MRv2. The number of enterprise applications (which means they have nice UIs) that can be ported to MR2 is orders of magnitude more than listed. People should worry more about using the container APIs than porting the UIs. Again, I'd like to call your attention to the *fundamental wrongness* of your approach as you fail/refuse to see it. The most common use case of AM UI is for application users themselves, which is more than 99% of the cluster users. You're trying to force people to adopt a special way, and in a sense, raising the taxes of the 99% for the sake of 1% (the self entitlement permeates throughout your arguments). Sorry to use a cliché but I was a 1% and I support the 99% :) bq. So Wiki/Twiki are a bad idea? Because aren't they a trusted front-end with a special interpreter for a special scheme? I can't speak for other wikis, but mediawiki (the one that powers wikipedia) had/has so many security bugs that I lost count. Many of the bugs are, not surprisingly, in the template interpreter and related code. bq. because I would never go to Wikipedia if I thought I could easily get a virus from it. This is a classic example of false sense of security through ignorance. You can get a virus from wikipedia, even if javascript is turned off, especially if you use a common combination of browsers/operating systems versions. I don't think you're not paranoid enough to write secure code yet :) bq. Didn't you say that you don't trust Caja. Why then didn't we go with a different library? I don't trust Caja yet but its security track record is way better than mediawiki. The main issue of Caja is that it's currently too slow (multi-pass/stage compiler) for non-trivial pages and that it doesn't have an official release yet. The other solutions I mentioned are not surprisingly existing proxy solutions. I was actually thinking about creating a simple plugin interface to allow people to delegate to deployment chosen solution for common web security, when the request is not the owner of AM. bq. I have seen very very old version of pig that are no longer supported being run because there is no motivation to fix it. I'm not sure if you're intentionally trying to conflate the issues here. I was talking about the difficulties of writing *trusted* *webapps*. The example you gave is neither and very much off topic. The main thrust of my original design is that a simple proxy can obviate the need to write complex (bug prone) trusted webapps in our particular use cases. OTOH, I do see your perspective as an on-call dev, as I've been there as well. Users are always trying to minimize their own work and avoiding upgrade if things are working for them. That's life and being a control freak doesn't really help here. You should focus your energy to fix the framework with reasonable security and limits and let users work within the limits. Improve Application Master And Job History UI Security -- Key: MAPREDUCE-3231 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3231 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Attachments: AMWebSecurityProposal.pdf I propose a stripped down JSON based protocol for creating safe user generate web pages. This JIRA is intended first of all as a place for a discussion about this proposal, and then if there are no serious objections this will be an Umbrella JIRA to implement the changes proposed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-3262: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Looks good to me too. I just committed this to trunk and branch-0.23. Thanks Hitesh and Sid! A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140073#comment-13140073 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Hdfs-0.23-Commit #111 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/111/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. svn merge -c r1195416 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195417 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140075#comment-13140075 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Common-0.23-Commit #111 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/111/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. svn merge -c r1195416 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195417 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140079#comment-13140079 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Hdfs-trunk-Commit #1280 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1280/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195416 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140077#comment-13140077 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Common-trunk-Commit #1204 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1204/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195416 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140083#comment-13140083 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1230 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1230/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195416 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140086#comment-13140086 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Mapreduce-0.23-Commit #120 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/120/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. svn merge -c r1195416 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195417 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3316) Rebooted link is not working properly
Rebooted link is not working properly
-
Key: MAPREDUCE-3316
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3316
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.0
Environment: NA
Reporter: Bhallamudi Venkata Siva Kamesh
Fix For: 0.23.0
While clicking on the *Rebooted Nodes* link, it is showing the following error
message
{color:red}Sorry, got error 500{color}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3316) Rebooted link is not working properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-3316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3316:
--
Attachment: reboot.png
Attaching the snapshot
Rebooted link is not working properly
-
Key: MAPREDUCE-3316
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3316
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.0
Environment: NA
Reporter: Bhallamudi Venkata Siva Kamesh
Fix For: 0.23.0
Attachments: reboot.png
While clicking on the *Rebooted Nodes* link, it is showing the following
error message
{color:red}Sorry, got error 500{color}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3316) Rebooted link is not working properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-3316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3316:
--
Attachment: MapReduce-3316.patch
Attaching the stack trace.
{noformat}
Caused by: java.lang.IllegalArgumentException: No enum const class
org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeState.REBOOTED
at java.lang.Enum.valueOf(Enum.java:196)
at
org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeState.valueOf(RMNodeState.java:21)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:82)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:64)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74)
at org.apache.hadoop.yarn.webapp.View.render(View.java:233)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:47)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:843)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:54)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:80)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:210)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:132)
{noformat}
problem is *REBOOTED* state has not been added to RMNodeState. Attaching the
patch for the same.
Rebooted link is not working properly
-
Key: MAPREDUCE-3316
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3316
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.0
Environment: NA
Reporter: Bhallamudi Venkata Siva Kamesh
Fix For: 0.23.0
Attachments: MapReduce-3316.patch, reboot.png
While clicking on the *Rebooted Nodes* link, it is showing the following
error message
{color:red}Sorry, got error 500{color}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140103#comment-13140103 ] Hudson commented on MAPREDUCE-3313: --- Integrated in Hadoop-Hdfs-0.23-Build #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/56/]) MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing TestResourceTrackerService sometimes. Contributed by Hitesh Shah. svn merge -c r1195319 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195320 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java TestResourceTrackerService failing in trunk some times -- Key: MAPREDUCE-3313 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, test Affects Versions: 0.23.0 Reporter: Ravi Gummadi Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3313.1.patch, MR-3313.2.patch TestResourceTrackerService is failing in trunk sometimes with the following error: testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) Time elapsed: 0.876 sec ERROR! java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) at org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140105#comment-13140105 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Hdfs-0.23-Build #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/56/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. svn merge -c r1195416 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195417 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3171) normalize nodemanager native code compilation with common/hdfs native
[ https://issues.apache.org/jira/browse/MAPREDUCE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140107#comment-13140107 ] Hudson commented on MAPREDUCE-3171: --- Integrated in Hadoop-Hdfs-0.23-Build #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/56/]) MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195257 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makefile.am * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c normalize nodemanager native code compilation with common/hdfs native - Key: MAPREDUCE-3171 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3171 Project: Hadoop Map/Reduce Issue Type: Improvement Components: build Affects Versions: 0.23.0, 0.24.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 0.23.0, 0.24.0 Attachments: MAPREDUCE-3171.patch, MAPREDUCE-3171.sh Use same build pattern as used by common/hdfs native: * rename src/c to src/native * run autoreconf, configure and make under target not to pollute the src tree * use maven-make-plugin in an identical way as in common/hdfs native -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140104#comment-13140104 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Hdfs-0.23-Build #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/56/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. svn merge -c r1195340 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195341 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3274) Race condition in MR App Master Preemtion can cause a dead lock
[ https://issues.apache.org/jira/browse/MAPREDUCE-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140106#comment-13140106 ] Hudson commented on MAPREDUCE-3274: --- Integrated in Hadoop-Hdfs-0.23-Build #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/56/]) MAPREDUCE-3274. Fixed a race condition in MRAppMaster that was causing a task-scheduling deadlock. Contributed by Robert Joseph Evans. svn merge -c r1195145 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195146 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/TaskAttemptListenerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/TaskAttemptListener.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java Race condition in MR App Master Preemtion can cause a dead lock --- Key: MAPREDUCE-3274 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3274 Project: Hadoop Map/Reduce Issue Type: Bug Components: applicationmaster, mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Priority: Blocker Fix For: 0.23.0, 0.24.0 Attachments: MR-3274.txt, MR-3274.txt There appears to be a race condition in the MR App Master in relation to preempting reducers to let a mapper run. In the particular case that I have been debugging a reducer was selected for preemption that did not have a container assigned to it yet. When the container became available that reduce started running and the previous TA_KILL event appears to have been ignored. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3240) NM should send a SIGKILL for completed containers also
[ https://issues.apache.org/jira/browse/MAPREDUCE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140110#comment-13140110 ] Hudson commented on MAPREDUCE-3240: --- Integrated in Hadoop-Hdfs-0.23-Build #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/56/]) MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195257 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makefile.am * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c NM should send a SIGKILL for completed containers also -- Key: MAPREDUCE-3240 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3240 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-3240-20111027.1.txt, MAPREDUCE-3240-20111027.2.txt, MR-3240.1.patch, MR-3240.2.patch, MR-3240.3.patch, MR-3240.4.wip.patch, MR-3240.5.patch, MR-3240.wip.patch This is to address the containers which exit properly after spawning sub-processes themselves. We don't want to leave these sub-process-tree or else they can pillage the NM's resources. Today, we already have code to send SIGKILL to the whole process-trees (because of single sessionId resulting from setsid) when the container is alive. We need to obtain the PID of the containers when they start and use that PID to send signal for completed containers' case also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2747) [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources
[ https://issues.apache.org/jira/browse/MAPREDUCE-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140109#comment-13140109 ] Hudson commented on MAPREDUCE-2747: --- Integrated in Hadoop-Hdfs-0.23-Build #56 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/56/]) MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195257 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makefile.am * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources - Key: MAPREDUCE-2747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2747 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager, security Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Robert Joseph Evans Priority: Blocker Fix For: 0.23.0, 0.24.0 Attachments: MAPREDUCE-2747-20111024-branch-0.23.txt, MAPREDUCE-2747-20111024.txt, MAPREDUCE-2747.patch, MR-2747-branch-0.23.txt, MR-2747.txt, MR-2747.txt, MR-2747.txt There are a lot of references to the old task-controller nomenclature still, job/task refs instead of app/container. Also the configuration file is named as taskcontroller.cfg and the configured variables are also from the mapred world (mrv1). These SHOULD be fixed before we make a release. Marking this as blocker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppStartedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3035) MR V2 jobhistory does not contain rack information
[ https://issues.apache.org/jira/browse/MAPREDUCE-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140113#comment-13140113 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3035: This one stalled a bit. I am taking this one from here to completion. MR V2 jobhistory does not contain rack information -- Key: MAPREDUCE-3035 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3035 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Karam Singh Assignee: chackaravarthy Priority: Critical Fix For: 0.23.1 Attachments: MAPREDUCE-3035-1.patch, MAPREDUCE-3035.patch When topology.node.switch.mapping.impl is set to enable rack-locality resolution via the topology script, from the RM web-UI, we can see the rack information for each node. Running a job also reveals the information about rack-local map tasks launched at end of job completion on the client side. But the hostname field for attempts in the JobHistory does not contain this rack information. In case of hadoop-0.20 securiy or MRV1, hostname field of job history does contain rackid/hostname whereas in MRV2, hostname field only contains the hostIP. Thus this is a regression. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140115#comment-13140115 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Hdfs-trunk #849 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/849/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195340 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/hadoop/mapreduce/v2 * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140114#comment-13140114 ] Hudson commented on MAPREDUCE-3313: --- Integrated in Hadoop-Hdfs-trunk #849 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/849/]) MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing TestResourceTrackerService sometimes. Contributed by Hitesh Shah. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195319 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java TestResourceTrackerService failing in trunk some times -- Key: MAPREDUCE-3313 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, test Affects Versions: 0.23.0 Reporter: Ravi Gummadi Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3313.1.patch, MR-3313.2.patch TestResourceTrackerService is failing in trunk sometimes with the following error: testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) Time elapsed: 0.876 sec ERROR! java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) at org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140116#comment-13140116 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Hdfs-trunk #849 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/849/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195416 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140141#comment-13140141 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Mapreduce-0.23-Build #76 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/76/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. svn merge -c r1195416 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195417 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3171) normalize nodemanager native code compilation with common/hdfs native
[ https://issues.apache.org/jira/browse/MAPREDUCE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140142#comment-13140142 ] Hudson commented on MAPREDUCE-3171: --- Integrated in Hadoop-Mapreduce-0.23-Build #76 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/76/]) MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195257 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makefile.am * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c normalize nodemanager native code compilation with common/hdfs native - Key: MAPREDUCE-3171 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3171 Project: Hadoop Map/Reduce Issue Type: Improvement Components: build Affects Versions: 0.23.0, 0.24.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 0.23.0, 0.24.0 Attachments: MAPREDUCE-3171.patch, MAPREDUCE-3171.sh Use same build pattern as used by common/hdfs native: * rename src/c to src/native * run autoreconf, configure and make under target not to pollute the src tree * use maven-make-plugin in an identical way as in common/hdfs native -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140139#comment-13140139 ] Hudson commented on MAPREDUCE-3313: --- Integrated in Hadoop-Mapreduce-0.23-Build #76 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/76/]) MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing TestResourceTrackerService sometimes. Contributed by Hitesh Shah. svn merge -c r1195319 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195320 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java TestResourceTrackerService failing in trunk some times -- Key: MAPREDUCE-3313 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, test Affects Versions: 0.23.0 Reporter: Ravi Gummadi Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3313.1.patch, MR-3313.2.patch TestResourceTrackerService is failing in trunk sometimes with the following error: testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) Time elapsed: 0.876 sec ERROR! java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) at org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/protocol/ClientProtocol.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2 * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3240) NM should send a SIGKILL for completed containers also
[ https://issues.apache.org/jira/browse/MAPREDUCE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140145#comment-13140145 ] Hudson commented on MAPREDUCE-3240: --- Integrated in Hadoop-Mapreduce-0.23-Build #76 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/76/]) MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195257 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makefile.am * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c NM should send a SIGKILL for completed containers also -- Key: MAPREDUCE-3240 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3240 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-3240-20111027.1.txt, MAPREDUCE-3240-20111027.2.txt, MR-3240.1.patch, MR-3240.2.patch, MR-3240.3.patch, MR-3240.4.wip.patch, MR-3240.5.patch, MR-3240.wip.patch This is to address the containers which exit properly after spawning sub-processes themselves. We don't want to leave these sub-process-tree or else they can pillage the NM's resources. Today, we already have code to send SIGKILL to the whole process-trees (because of single sessionId resulting from setsid) when the container is alive. We need to obtain the PID of the containers when they start and use that PID to send signal for completed containers' case also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2747) [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources
[ https://issues.apache.org/jira/browse/MAPREDUCE-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140144#comment-13140144 ] Hudson commented on MAPREDUCE-2747: --- Integrated in Hadoop-Mapreduce-0.23-Build #76 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/76/]) MAPREDUCE-3171 merge from trunk reverted changes from MAPREDUCE-2747 MAPREDUCE-3240. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195257 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makefile.am * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources - Key: MAPREDUCE-2747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2747 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager, security Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Robert Joseph Evans Priority: Blocker Fix For: 0.23.0, 0.24.0 Attachments: MAPREDUCE-2747-20111024-branch-0.23.txt, MAPREDUCE-2747-20111024.txt, MAPREDUCE-2747.patch, MR-2747-branch-0.23.txt, MR-2747.txt, MR-2747.txt, MR-2747.txt There are a lot of references to the old task-controller nomenclature still, job/task refs instead of app/container. Also the configuration file is named as taskcontroller.cfg and the configured variables are also from the mapred world (mrv1). These SHOULD be fixed before we make a release. Marking this as blocker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerAppStartedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3313) TestResourceTrackerService failing in trunk some times
[ https://issues.apache.org/jira/browse/MAPREDUCE-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140152#comment-13140152 ] Hudson commented on MAPREDUCE-3313: --- Integrated in Hadoop-Mapreduce-trunk #883 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/883/]) MAPREDUCE-3313. Fixed initialization of ClusterMetrics which was failing TestResourceTrackerService sometimes. Contributed by Hitesh Shah. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195319 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java TestResourceTrackerService failing in trunk some times -- Key: MAPREDUCE-3313 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3313 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, test Affects Versions: 0.23.0 Reporter: Ravi Gummadi Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3313.1.patch, MR-3313.2.patch TestResourceTrackerService is failing in trunk sometimes with the following error: testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) Time elapsed: 0.876 sec ERROR! java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.ClusterMetrics.getNumDecommisionedNMs(ClusterMetrics.java:78) at org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3146) Add a MR specific command line to dump logs for a given TaskAttemptID
/org/apache/hadoop/mapreduce/v2 * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/v2/LogParams.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithHistoryService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/LogDumper.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTracker.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LocalJobRunner.java Add a MR specific command line to dump logs for a given TaskAttemptID - Key: MAPREDUCE-3146 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3146 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli Assignee: Siddharth Seth Priority: Critical Fix For: 0.23.0 Attachments: MAPREDUCE-3146-20111031.txt, MR3146_v1.txt, MR3146_v2.txt, MR3146_v3.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140153#comment-13140153 ] Hudson commented on MAPREDUCE-2766: --- Integrated in Hadoop-Mapreduce-trunk #883 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/883/]) MAPREDUCE-2766. Fixed NM to set secure permissions for files and directories in distributed-cache. Contributed by Hitesh Shah. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195340 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/FSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestFSDownload.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java [MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MAPREDUCE-2766-20111031.txt, MR-2766.1.patch Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3262) A few events are not handled by the NodeManager in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140154#comment-13140154 ] Hudson commented on MAPREDUCE-3262: --- Integrated in Hadoop-Mapreduce-trunk #883 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/883/]) MAPREDUCE-3262. Fixed Container's state-machine in NodeManager to handle a couple of events in failure states correctly. Contributed by Hitesh Shah and Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1195416 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java A few events are not handled by the NodeManager in failure scenarios Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical Fix For: 0.23.0 Attachments: MR-3262.1.patch, MR3262.txt Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2696) Container logs aren't getting cleaned up when LogAggregation is disabled
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerContainerFinishedEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/event/LogHandlerEventType.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/AggregatedLogsBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/loghandler/TestNonAggregatingLogHandler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm Container logs aren't getting cleaned up when LogAggregation is disabled Key: MAPREDUCE-2696 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2696 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Siddharth Seth Fix For: 0.23.0 Attachments: MAPREDUCE-2696-20111031.1.txt, MAPREDUCE-2696-20111031.txt, MR2696_v1.txt, MR2696_v1.txt Container logs aren't getting cleaned up when log-aggregation is disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3231) Improve Application Master And Job History UI Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-3231: --- Assignee: Luke Lu (was: Robert Joseph Evans) @Luke I obviously haven't convinced you. I have assigned the JIRA to you so you can do with it as you see fit. Feel free to close it or use it to make the changes that you wanted on it. Improve Application Master And Job History UI Security -- Key: MAPREDUCE-3231 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3231 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Robert Joseph Evans Assignee: Luke Lu Attachments: AMWebSecurityProposal.pdf I propose a stripped down JSON based protocol for creating safe user generate web pages. This JIRA is intended first of all as a place for a discussion about this proposal, and then if there are no serious objections this will be an Umbrella JIRA to implement the changes proposed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3275) Add docs for WebAppProxy
[ https://issues.apache.org/jira/browse/MAPREDUCE-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-3275: --- Attachment: MR-3275.txt Addressed the comments so far. Sorry it took me so long to get this one in. Add docs for WebAppProxy Key: MAPREDUCE-3275 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3275 Project: Hadoop Map/Reduce Issue Type: Improvement Components: documentation, mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Priority: Critical Fix For: 0.24.0 Attachments: MR-3275.txt, MR-3275.txt In my haste to get the WebAppProxy code in the documentation for it was neglected. This is to fix that. Docs need to be added to ClusterSetup.html about how to configure and use the WebAppProxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3275) Add docs for WebAppProxy
[ https://issues.apache.org/jira/browse/MAPREDUCE-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-3275: --- Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) Add docs for WebAppProxy Key: MAPREDUCE-3275 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3275 Project: Hadoop Map/Reduce Issue Type: Improvement Components: documentation, mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Priority: Critical Fix For: 0.24.0 Attachments: MR-3275.txt, MR-3275.txt In my haste to get the WebAppProxy code in the documentation for it was neglected. This is to fix that. Docs need to be added to ClusterSetup.html about how to configure and use the WebAppProxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3275) Add docs for WebAppProxy
[ https://issues.apache.org/jira/browse/MAPREDUCE-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13140198#comment-13140198 ] Hadoop QA commented on MAPREDUCE-3275: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501612/MR-3275.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1220//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1220//console This message is automatically generated. Add docs for WebAppProxy Key: MAPREDUCE-3275 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3275 Project: Hadoop Map/Reduce Issue Type: Improvement Components: documentation, mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Priority: Critical Fix For: 0.24.0 Attachments: MR-3275.txt, MR-3275.txt In my haste to get the WebAppProxy code in the documentation for it was neglected. This is to fix that. Docs need to be added to ClusterSetup.html about how to configure and use the WebAppProxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3246) Make Task extensible to support modifications of Task or even alternate programming paradigms
[ https://issues.apache.org/jira/browse/MAPREDUCE-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Binglin Chang updated MAPREDUCE-3246: - Assignee: Binglin Chang Status: Patch Available (was: Open) Changes: # JvmTask readFields/write with class name # Replace Task constructors with parameters to init() methods # Add jobconf: mapreduce.map.task.class mapreduce.reduce.task.class Make Task extensible to support modifications of Task or even alternate programming paradigms - Key: MAPREDUCE-3246 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3246 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Binglin Chang Assignee: Binglin Chang Attachments: MAPREDUCE-3246-extensible-task.patch One of MRv2's goal is to support alternate programming paradigms, but building a application using YARN from the bottom is not trivial. In fact most component of MapReduce can be reused, mostly the scheduler/master side, and we can make changes/extensions only on the task/slave side, such as native tasks, hash-aggregation style combiner/reducer interfaces. The first thing to do I think is to make task/slave side extensible, more specific, the Task in JvmTask should serialized with class name, not simply a boolean isMap, and make task class name configurable in JobConf, there maybe other minor changes. By doing so, developers can at least extends their own MapTask/ReduceTask. I just post my initial thoughts here for opinions. If this change is OK, I can submit a patch, this is just a trivial work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
