[jira] [Created] (MAPREDUCE-2640) The maxRunningTasks property of the LimitTasksPerJob scheduler is ambiguous in its description, and must be updated
The maxRunningTasks property of the LimitTasksPerJob scheduler is ambiguous in
its description, and must be updated
---
Key: MAPREDUCE-2640
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2640
Project: Hadoop Map/Reduce
Issue Type: Task
Components: documentation
Affects Versions: 0.20.2
Reporter: Harsh J
Assignee: Harsh J
Fix For: 0.23.0
The property's entry in mapred-default.xml is like so:
{code}
property
namemapreduce.jobtracker.taskscheduler.maxrunningtasks.perjob/name
value/value
descriptionThe maximum number of running tasks for a job before
it gets preempted. No limits if undefined.
/description
/property
{code}
There is no mention that this is a property exclusive to the LimitTasksPerJob
scheduler alone. The doc ought to be updated to note that unless there's a plan
of reusing such a property (I do not see fair or capacity schedulers utilizing
this, and they use their own configs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2639) MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Wills updated MAPREDUCE-2639: -- Attachment: (was: MAPREDUCE-2639.patch) MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup. --- Key: MAPREDUCE-2639 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2639 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Environment: All Reporter: Josh Wills Assignee: Josh Wills Priority: Minor A catch-all JIRA for a pass I took through the v2.app.speculate package. 1) Fixed the ExponentiallySmoothedTaskRuntimeEstimator so that it can run and pass the test defined in TestRuntimeEstimators. 2) Fixed two bugs in DataStatistics: 1) a divide by zero in the variance calculation in the case that count == 0 and 2) a synchronization issue in how the updateStatistics method was implemented, 3) A bunch of typo corrections, formatting fixes, and adding some consistency around the null value checking. I probably need to do a couple more passes through this code to get it into better shape, but this seemed like a good start. Will attach my patch momentarily. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2639) MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Wills updated MAPREDUCE-2639: -- Attachment: MAPREDUCE-2639-2.patch Bug fixes for the DataStatistics class and the test I added. MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup. --- Key: MAPREDUCE-2639 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2639 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Environment: All Reporter: Josh Wills Assignee: Josh Wills Priority: Minor Attachments: MAPREDUCE-2639-1.patch, MAPREDUCE-2639-2.patch, MAPREDUCE-2639-3.patch A catch-all JIRA for a pass I took through the v2.app.speculate package. 1) Fixed the ExponentiallySmoothedTaskRuntimeEstimator so that it can run and pass the test defined in TestRuntimeEstimators. 2) Fixed two bugs in DataStatistics: 1) a divide by zero in the variance calculation in the case that count == 0 and 2) a synchronization issue in how the updateStatistics method was implemented, 3) A bunch of typo corrections, formatting fixes, and adding some consistency around the null value checking. I probably need to do a couple more passes through this code to get it into better shape, but this seemed like a good start. Will attach my patch momentarily. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2639) MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Wills updated MAPREDUCE-2639: -- Attachment: MAPREDUCE-2639-3.patch Typo correction, formatting/naming fixes, and consistent null handling in the other classes in v2.app.speculate. No non-trivial functional changes. MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup. --- Key: MAPREDUCE-2639 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2639 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Environment: All Reporter: Josh Wills Assignee: Josh Wills Priority: Minor Attachments: MAPREDUCE-2639-1.patch, MAPREDUCE-2639-2.patch, MAPREDUCE-2639-3.patch A catch-all JIRA for a pass I took through the v2.app.speculate package. 1) Fixed the ExponentiallySmoothedTaskRuntimeEstimator so that it can run and pass the test defined in TestRuntimeEstimators. 2) Fixed two bugs in DataStatistics: 1) a divide by zero in the variance calculation in the case that count == 0 and 2) a synchronization issue in how the updateStatistics method was implemented, 3) A bunch of typo corrections, formatting fixes, and adding some consistency around the null value checking. I probably need to do a couple more passes through this code to get it into better shape, but this seemed like a good start. Will attach my patch momentarily. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2639) MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13059644#comment-13059644 ] Aaron T. Myers commented on MAPREDUCE-2639: --- Hey Josh, I'd recommend breaking this out into separate JIRAs as subtasks of this one, so that those who want to give feedback can localize the review/discussion to smaller areas. MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup. --- Key: MAPREDUCE-2639 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2639 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Environment: All Reporter: Josh Wills Assignee: Josh Wills Priority: Minor Attachments: MAPREDUCE-2639-1.patch, MAPREDUCE-2639-2.patch, MAPREDUCE-2639-3.patch A catch-all JIRA for a pass I took through the v2.app.speculate package. 1) Fixed the ExponentiallySmoothedTaskRuntimeEstimator so that it can run and pass the test defined in TestRuntimeEstimators. 2) Fixed two bugs in DataStatistics: 1) a divide by zero in the variance calculation in the case that count == 0 and 2) a synchronization issue in how the updateStatistics method was implemented, 3) A bunch of typo corrections, formatting fixes, and adding some consistency around the null value checking. I probably need to do a couple more passes through this code to get it into better shape, but this seemed like a good start. Will attach my patch momentarily. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
