[jira] [Updated] (MAPREDUCE-3008) [Gridmix] Improve cumulative CPU usage emulation for short running tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-3008: Fix Version/s: (was: 2.1.0-beta) (was: 3.0.0) [Gridmix] Improve cumulative CPU usage emulation for short running tasks Key: MAPREDUCE-3008 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3008 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: cpu-emulation, gridmix Fix For: 1.1.0 Attachments: mapreduce-2591-v1.4.2.patch, mapreduce-2591-v1.7-0.20-security.patch, mapreduce-2591-v1.7.patch CPU emulation in Gridmix fails to meet the expected target if the map has no data to sort/spill/merge. There are 2 major reasons for this: 1. The map task end immediately ends soon after the map task. The map progress is 67% while the map phase ends. 2. Currently, the sort (comparator) doesnt emulate CPU. If the map is short lived, the CPU emulation thread (spawned from the map task in cleanup) doesn't get a chance to emulate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3008) [Gridmix] Improve cumulative CPU usage emulation for short running tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated MAPREDUCE-3008: -- Target Version/s: (was: 0.23.1) Fix Version/s: 1.1.0 [Gridmix] Improve cumulative CPU usage emulation for short running tasks Key: MAPREDUCE-3008 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3008 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: cpu-emulation, gridmix Fix For: 1.1.0, 3.0.0 Attachments: mapreduce-2591-v1.4.2.patch, mapreduce-2591-v1.7-0.20-security.patch, mapreduce-2591-v1.7.patch CPU emulation in Gridmix fails to meet the expected target if the map has no data to sort/spill/merge. There are 2 major reasons for this: 1. The map task end immediately ends soon after the map task. The map progress is 67% while the map phase ends. 2. Currently, the sort (comparator) doesnt emulate CPU. If the map is short lived, the CPU emulation thread (spawned from the map task in cleanup) doesn't get a chance to emulate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3008) [Gridmix] Improve cumulative CPU usage emulation for short running tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated MAPREDUCE-3008: --- Target Version/s: 0.23.1 [Gridmix] Improve cumulative CPU usage emulation for short running tasks Key: MAPREDUCE-3008 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3008 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: cpu-emulation, gridmix Fix For: 0.24.0 Attachments: mapreduce-2591-v1.4.2.patch, mapreduce-2591-v1.7-0.20-security.patch, mapreduce-2591-v1.7.patch CPU emulation in Gridmix fails to meet the expected target if the map has no data to sort/spill/merge. There are 2 major reasons for this: 1. The map task end immediately ends soon after the map task. The map progress is 67% while the map phase ends. 2. Currently, the sort (comparator) doesnt emulate CPU. If the map is short lived, the CPU emulation thread (spawned from the map task in cleanup) doesn't get a chance to emulate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3008) [Gridmix] Improve cumulative CPU usage emulation for short running tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat updated MAPREDUCE-3008: -- Attachment: mapreduce-2591-v1.7-0.20-security.patch Backported patch for the branch-0.20-security branch. System tests passed. Our analysis shows improvements in emulation. I couldn't run test-patch and JUnit tests as the code compilation fails because of task-controller. [Gridmix] Improve cumulative CPU usage emulation for short running tasks Key: MAPREDUCE-3008 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3008 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Assignee: Amar Kamat Labels: cpu-emulation, gridmix Fix For: 0.24.0 Attachments: mapreduce-2591-v1.4.2.patch, mapreduce-2591-v1.7-0.20-security.patch, mapreduce-2591-v1.7.patch CPU emulation in Gridmix fails to meet the expected target if the map has no data to sort/spill/merge. There are 2 major reasons for this: 1. The map task end immediately ends soon after the map task. The map progress is 67% while the map phase ends. 2. Currently, the sort (comparator) doesnt emulate CPU. If the map is short lived, the CPU emulation thread (spawned from the map task in cleanup) doesn't get a chance to emulate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3008) [Gridmix] Improve cumulative CPU usage emulation for short running tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amar Kamat updated MAPREDUCE-3008:
--
Attachment: mapreduce-2591-v1.7.patch
Attaching a new patch incorporating Ravi's offline comments.
test-patch passed:
{noformat}
+1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
{noformat}
All the JUnit tests in test-contrib passed except TestUserResolve, which is a
known issue (MAPREDUCE-2950).
[Gridmix] Improve cumulative CPU usage emulation for short running tasks
Key: MAPREDUCE-3008
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3008
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: contrib/gridmix
Affects Versions: 0.24.0
Reporter: Amar Kamat
Labels: cpu-emulation, gridmix
Fix For: 0.24.0
Attachments: mapreduce-2591-v1.4.2.patch, mapreduce-2591-v1.7.patch
CPU emulation in Gridmix fails to meet the expected target if the map has no
data to sort/spill/merge. There are 2 major reasons for this:
1. The map task end immediately ends soon after the map task. The map
progress is 67% while the map phase ends.
2. Currently, the sort (comparator) doesnt emulate CPU. If the map is short
lived, the CPU emulation thread (spawned from the map task in cleanup)
doesn't get a chance to emulate.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3008) [Gridmix] Improve cumulative CPU usage emulation for short running tasks
[ https://issues.apache.org/jira/browse/MAPREDUCE-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat updated MAPREDUCE-3008: -- Attachment: mapreduce-2591-v1.4.2.patch Attaching a patch that improves CPU emulation for short running tasks. Areas of improvements: 1. Sorter/Comparator now is CPU emulation aware 2. For tasks with no spills/merges, aggressive CPU emulation is done. tets-patch and JUnit tests for Gridmix passed. [Gridmix] Improve cumulative CPU usage emulation for short running tasks Key: MAPREDUCE-3008 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3008 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: contrib/gridmix Affects Versions: 0.24.0 Reporter: Amar Kamat Labels: cpu-emulation, gridmix Fix For: 0.24.0 Attachments: mapreduce-2591-v1.4.2.patch CPU emulation in Gridmix fails to meet the expected target if the map has no data to sort/spill/merge. There are 2 major reasons for this: 1. The map task end immediately ends soon after the map task. The map progress is 67% while the map phase ends. 2. Currently, the sort (comparator) doesnt emulate CPU. If the map is short lived, the CPU emulation thread (spawned from the map task in cleanup) doesn't get a chance to emulate. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
