[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258899#comment-13258899 ] Harsh J commented on MAPREDUCE-3812: Didn't realize this patch changed the way allocs would get read, or that it was doing more than just changing default AM memory requirements. Would save me a lot of work I've otherwise been doing on MAPREDUCE-4026 and MAPREDUCE-4027. I'll take over this one since it looks to have gone stale. Thanks for revamping the whole default way of reading the allocs - much simpler way now. No idea why there was per-scheduler configs before when you can't use more than one scheduler anyway. > Change default memory slot sizes to be 1.5GB > > > Key: MAPREDUCE-3812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3812-20120205.txt, > MAPREDUCE-3812-20120206.1.txt, MAPREDUCE-3812-20120206.txt, > MAPREDUCE-3812.patch, MAPREDUCE-3812.patch > > > After a few performance improvements tracked at MAPREDUCE-3561, like > MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within > 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work > around the issues with AM heap. Now that those are fixed, we should go back > to 1GB. > This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202089#comment-13202089 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513566/MAPREDUCE-3812.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestMiniMRBringup org.apache.hadoop.mapred.TestMiniMRChildTask org.apache.hadoop.mapred.TestReduceFetch org.apache.hadoop.mapred.TestReduceFetchFromPartialMem org.apache.hadoop.mapred.TestJobCounters org.apache.hadoop.mapreduce.TestChild org.apache.hadoop.mapred.TestMiniMRClientCluster org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter org.apache.hadoop.mapred.TestClientRedirect org.apache.hadoop.mapred.TestLazyOutput org.apache.hadoop.mapreduce.TestMapReduceLazyOutput org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath org.apache.hadoop.mapreduce.v2.TestRMNMInfo org.apache.hadoop.mapred.TestJobSysDirWithDFS org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1809//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1809//console This message is automatically generated. > Change default memory slot sizes to be 1.5GB > > > Key: MAPREDUCE-3812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3812-20120205.txt, > MAPREDUCE-3812-20120206.1.txt, MAPREDUCE-3812-20120206.txt, > MAPREDUCE-3812.patch, MAPREDUCE-3812.patch > > > After a few performance improvements tracked at MAPREDUCE-3561, like > MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within > 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work > around the issues with AM heap. Now that those are fixed, we should go back > to 1GB. > This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201995#comment-13201995 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513549/MAPREDUCE-3812.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1807//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1807//console This message is automatically generated. > Change default memory slot sizes to be 1.5GB > > > Key: MAPREDUCE-3812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3812-20120205.txt, > MAPREDUCE-3812-20120206.1.txt, MAPREDUCE-3812-20120206.txt, > MAPREDUCE-3812.patch > > > After a few performance improvements tracked at MAPREDUCE-3561, like > MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within > 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work > around the issues with AM heap. Now that those are fixed, we should go back > to 1GB. > This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201706#comment-13201706 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513504/MAPREDUCE-3812-20120206.1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1799//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1799//console This message is automatically generated. > Change default memory slot sizes to be 1.5GB > > > Key: MAPREDUCE-3812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3812-20120205.txt, > MAPREDUCE-3812-20120206.1.txt, MAPREDUCE-3812-20120206.txt > > > After a few performance improvements tracked at MAPREDUCE-3561, like > MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within > 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work > around the issues with AM heap. Now that those are fixed, we should go back > to 1GB. > This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201534#comment-13201534 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513476/MAPREDUCE-3812-20120206.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup org.apache.hadoop.yarn.server.resourcemanager.TestFifoScheduler org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1791//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1791//console This message is automatically generated. > Change default memory slot sizes to be 1.5GB > > > Key: MAPREDUCE-3812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3812-20120205.txt, MAPREDUCE-3812-20120206.txt > > > After a few performance improvements tracked at MAPREDUCE-3561, like > MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within > 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work > around the issues with AM heap. Now that those are fixed, we should go back > to 1GB. > This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200978#comment-13200978 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1782//console This message is automatically generated. > Change default memory slot sizes to be 1.5GB > > > Key: MAPREDUCE-3812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3812-20120205.txt > > > After a few performance improvements tracked at MAPREDUCE-3561, like > MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within > 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work > around the issues with AM heap. Now that those are fixed, we should go back > to 1GB. > This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
[ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200976#comment-13200976 ] Hadoop QA commented on MAPREDUCE-3812: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513379/MAPREDUCE-3812-20120205.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1781//console This message is automatically generated. > Change default memory slot sizes to be 1.5GB > > > Key: MAPREDUCE-3812 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, performance >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.1 > > Attachments: MAPREDUCE-3812-20120205.txt > > > After a few performance improvements tracked at MAPREDUCE-3561, like > MAPREDUCE-3511 and MAPREDUCE-3567, even a 100K maps job can also run within > 1GB vmem. We earlier increased AM slot size from 1 slot to two slots to work > around the issues with AM heap. Now that those are fixed, we should go back > to 1GB. > This is just a configuration change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
