[jira] [Commented] (MAPREDUCE-3787) [Gridmix] Improve STRESS mode
[ https://issues.apache.org/jira/browse/MAPREDUCE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214524#comment-13214524 ] Ravi Gummadi commented on MAPREDUCE-3787: - Patch looks good to me. +1 [Gridmix] Improve STRESS mode - Key: MAPREDUCE-3787 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3787 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, stress Fix For: 0.23.1 Attachments: MAPREDUCE-3787-v1.12.patch, MAPREDUCE-3787-v1.9.patch Gridmix STRESS mode can be improved as follows: 1. The sleep time in JobMonitor can be reduced and/or made configurable 2. Map and reduce load calculation in StressJobFactory can be done in one loop 3. Updating the overload status from the job submitter thread (inline) 4. Optimizations to avoid un-necessary progress check (which inturn would result into delay) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3787) [Gridmix] Improve STRESS mode
[ https://issues.apache.org/jira/browse/MAPREDUCE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214542#comment-13214542 ] Hudson commented on MAPREDUCE-3787: --- Integrated in Hadoop-Hdfs-trunk-Commit #1840 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1840/]) MAPREDUCE-3787. [Gridmix] Optimize job monitoring and STRESS mode for faster job submission. (amarrk) (Revision 1292736) Result = SUCCESS amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1292736 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/ExecutionSummarizer.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobMonitor.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Statistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/StressJobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixStatistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSubmission.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSummary.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestSleepJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/docs/src/documentation/content/xdocs/gridmix.xml [Gridmix] Improve STRESS mode - Key: MAPREDUCE-3787 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3787 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, stress Fix For: 0.23.1 Attachments: MAPREDUCE-3787-v1.12.patch, MAPREDUCE-3787-v1.9.patch Gridmix STRESS mode can be improved as follows: 1. The sleep time in JobMonitor can be reduced and/or made configurable 2. Map and reduce load calculation in StressJobFactory can be done in one loop 3. Updating the overload status from the job submitter thread (inline) 4. Optimizations to avoid un-necessary progress check (which inturn would result into delay) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3787) [Gridmix] Improve STRESS mode
[ https://issues.apache.org/jira/browse/MAPREDUCE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214545#comment-13214545 ] Hudson commented on MAPREDUCE-3787: --- Integrated in Hadoop-Common-trunk-Commit #1766 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1766/]) MAPREDUCE-3787. [Gridmix] Optimize job monitoring and STRESS mode for faster job submission. (amarrk) (Revision 1292736) Result = SUCCESS amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1292736 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/ExecutionSummarizer.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobMonitor.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Statistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/StressJobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixStatistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSubmission.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSummary.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestSleepJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/docs/src/documentation/content/xdocs/gridmix.xml [Gridmix] Improve STRESS mode - Key: MAPREDUCE-3787 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3787 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, stress Fix For: 0.23.1 Attachments: MAPREDUCE-3787-v1.12.patch, MAPREDUCE-3787-v1.9.patch Gridmix STRESS mode can be improved as follows: 1. The sleep time in JobMonitor can be reduced and/or made configurable 2. Map and reduce load calculation in StressJobFactory can be done in one loop 3. Updating the overload status from the job submitter thread (inline) 4. Optimizations to avoid un-necessary progress check (which inturn would result into delay) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3787) [Gridmix] Improve STRESS mode
[ https://issues.apache.org/jira/browse/MAPREDUCE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214559#comment-13214559 ] Hudson commented on MAPREDUCE-3787: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1777 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1777/]) MAPREDUCE-3787. [Gridmix] Optimize job monitoring and STRESS mode for faster job submission. (amarrk) (Revision 1292736) Result = ABORTED amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1292736 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/ExecutionSummarizer.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobMonitor.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Statistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/StressJobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixStatistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSubmission.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSummary.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestSleepJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/docs/src/documentation/content/xdocs/gridmix.xml [Gridmix] Improve STRESS mode - Key: MAPREDUCE-3787 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3787 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, stress Fix For: 0.23.1 Attachments: MAPREDUCE-3787-v1.12.patch, MAPREDUCE-3787-v1.9.patch Gridmix STRESS mode can be improved as follows: 1. The sleep time in JobMonitor can be reduced and/or made configurable 2. Map and reduce load calculation in StressJobFactory can be done in one loop 3. Updating the overload status from the job submitter thread (inline) 4. Optimizations to avoid un-necessary progress check (which inturn would result into delay) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3787) [Gridmix] Improve STRESS mode
[ https://issues.apache.org/jira/browse/MAPREDUCE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214584#comment-13214584 ] Hudson commented on MAPREDUCE-3787: --- Integrated in Hadoop-Hdfs-trunk #964 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/964/]) MAPREDUCE-3787. [Gridmix] Optimize job monitoring and STRESS mode for faster job submission. (amarrk) (Revision 1292736) Result = SUCCESS amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1292736 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/ExecutionSummarizer.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobMonitor.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Statistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/StressJobFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixStatistics.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSubmission.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestGridmixSummary.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestSleepJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/docs/src/documentation/content/xdocs/gridmix.xml [Gridmix] Improve STRESS mode - Key: MAPREDUCE-3787 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3787 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: gridmix, stress Fix For: 0.23.1 Attachments: MAPREDUCE-3787-v1.12.patch, MAPREDUCE-3787-v1.9.patch Gridmix STRESS mode can be improved as follows: 1. The sleep time in JobMonitor can be reduced and/or made configurable 2. Map and reduce load calculation in StressJobFactory can be done in one loop 3. Updating the overload status from the job submitter thread (inline) 4. Optimizations to avoid un-necessary progress check (which inturn would result into delay) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
