[jira] Commented: (MAPREDUCE-1710) Process tree clean up of exceeding memory limit tasks.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000236#comment-13000236 ] Hadoop QA commented on MAPREDUCE-1710: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449101/MAPREDUCE-1710.patch against trunk revision 1075216. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/84//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/84//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/84//console This message is automatically generated. > Process tree clean up of exceeding memory limit tasks. > -- > > Key: MAPREDUCE-1710 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1710 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: test >Affects Versions: 0.21.0 >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Attachments: 1710-ydist_security.patch, 1710-ydist_security.patch, > 1710-ydist_security.patch, MAPREDUCE-1710.patch, memorylimittask_1710.patch, > memorylimittask_1710.patch, memorylimittask_1710.patch, > memorylimittask_1710.patch, memorylimittask_1710.patch > > > 1. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Let the job complete . Check if all the > child processes are killed, the overall job should fail. > 2. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Kill/fail the job while in progress. > Check if all the child processes are killed. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1710) Process tree clean up of exceeding memory limit tasks.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890335#action_12890335 ] Vinay Kumar Thota commented on MAPREDUCE-1710: -- There are two failures that are not related to this patch. > Process tree clean up of exceeding memory limit tasks. > -- > > Key: MAPREDUCE-1710 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1710 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: test >Affects Versions: 0.21.0 >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Attachments: 1710-ydist_security.patch, 1710-ydist_security.patch, > 1710-ydist_security.patch, MAPREDUCE-1710.patch, memorylimittask_1710.patch, > memorylimittask_1710.patch, memorylimittask_1710.patch, > memorylimittask_1710.patch, memorylimittask_1710.patch > > > 1. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Let the job complete . Check if all the > child processes are killed, the overall job should fail. > 2. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Kill/fail the job while in progress. > Check if all the child processes are killed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1710) Process tree clean up of exceeding memory limit tasks.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888247#action_12888247 ] Hadoop QA commented on MAPREDUCE-1710: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449101/MAPREDUCE-1710.patch against trunk revision 962682. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/596/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/596/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/596/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/596/console This message is automatically generated. > Process tree clean up of exceeding memory limit tasks. > -- > > Key: MAPREDUCE-1710 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1710 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: test >Affects Versions: 0.21.0 >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Attachments: 1710-ydist_security.patch, 1710-ydist_security.patch, > 1710-ydist_security.patch, MAPREDUCE-1710.patch, memorylimittask_1710.patch, > memorylimittask_1710.patch, memorylimittask_1710.patch, > memorylimittask_1710.patch, memorylimittask_1710.patch > > > 1. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Let the job complete . Check if all the > child processes are killed, the overall job should fail. > 2. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Kill/fail the job while in progress. > Check if all the child processes are killed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1710) Process tree clean up of exceeding memory limit tasks.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860115#action_12860115 ] Iyappan Srinivasan commented on MAPREDUCE-1710: --- Since this contains only testcase now and no framework related changes, I have checked the latest patch of testcase and found it good. +1 > Process tree clean up of exceeding memory limit tasks. > -- > > Key: MAPREDUCE-1710 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1710 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: test >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Attachments: memorylimittask_1710.patch, memorylimittask_1710.patch > > > 1. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Let the job complete . Check if all the > child processes are killed, the overall job should fail. > 2. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Kill/fail the job while in progress. > Check if all the child processes are killed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1710) Process tree clean up of exceeding memory limit tasks.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858784#action_12858784 ] Vinay Kumar Thota commented on MAPREDUCE-1710: -- 3) resetClusterWithNewConfigMapred seems a better name than resetCluster, becuse users can understand the intent of this feature just by reading the name rather than seeing the javadocs. The same goes with resetCluster. It can be resetClusterWithOldConfig. [Vinay] : Even I agreed your point and also I thought of defining same kind of method names while developing,however I felt that restartCluster and resetCluster would be fine and there won't be a lengthy method name. Any how user can understand the behavior of method by reading the java docs. So I felt that there is not importance of putting lengthy method name. 5) if (!tttInfo.isTaskCleanupTask()) - Even setup task could be checked to avoid it and made fully sure its only mapper. [Vinay]: I didn't see any method for specific to setup cleanup in ttInfo. the method isTaskCleanupTask might be checking for both. Apart from above two comments, I have addressed all other comments and attached the latest patch. > Process tree clean up of exceeding memory limit tasks. > -- > > Key: MAPREDUCE-1710 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1710 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: test >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Attachments: memorylimittask_1710.patch > > > 1. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Let the job complete . Check if all the > child processes are killed, the overall job should fail. > 2. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Kill/fail the job while in progress. > Check if all the child processes are killed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1710) Process tree clean up of exceeding memory limit tasks.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858394#action_12858394 ] Iyappan Srinivasan commented on MAPREDUCE-1710: --- Code looks good. Some comments. 1) Check if all import statements make sense . - NetworkedJob is not required. Check for all imports. 2) There should be space between parameters in methods like in this : UtilsForSystemTests.restartCluster(cluster,prop); Check this for throughout testcase. 3) resetClusterWithNewConfigMapred seems a better name than resetCluster, becuse users can understand the intent of this feature just by reading the name rather than seeing the javadocs. The same goes with resetCluster. It can be resetClusterWithOldConfig. 4) All input, output directory creation and deletion should be done in BeforeClass and AfterClass. This to avoid testcase assuming that a directory is already present. and a testacse cleaning up all directories it created before exiting, even if it failed inteh middle. 5) if (!tttInfo.isTaskCleanupTask()) - Even setup task could be checked to avoid it and made fully sure its only mapper. > Process tree clean up of exceeding memory limit tasks. > -- > > Key: MAPREDUCE-1710 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1710 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: test >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Attachments: memorylimittask_1710.patch > > > 1. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Let the job complete . Check if all the > child processes are killed, the overall job should fail. > 2. Submit a job which would spawn child processes and each of the child > processes exceeds the memory limits. Kill/fail the job while in progress. > Check if all the child processes are killed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
