[jira] [Updated] (MAPREDUCE-4085) Kill task attempts longer than a configured queue max time
[
https://issues.apache.org/jira/browse/MAPREDUCE-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated MAPREDUCE-4085:
Attachment: MAPREDUCE-4085-branch-1.0.4.txt
Here's an updated version for anyone who wants it. This one also includes the
ability for users to set a smaller task time limit
(mapred.job.{map|reduce}.task-wallclock-limit) in case they want something
faster. i.e., I know my task should finish in 5 minutes, so kill it if it
doesn't. Of course, the queue time out will still kick in if the user
provided time is longer.
Kill task attempts longer than a configured queue max time
--
Key: MAPREDUCE-4085
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4085
Project: Hadoop Map/Reduce
Issue Type: New Feature
Components: task
Reporter: Allen Wittenauer
Attachments: MAPREDUCE-4085-branch-1.0.4.txt,
MAPREDUCE-4085-branch-1.0.txt
For some environments, it is desirable to have certain queues have an SLA
with regards to task turnover. (i.e., a slot will be free in X minutes and
scheduled to the appropriate job) Queues should have a 'task time limit'
that would cause task attempts over this time to be killed. This leaves open
the possibility that if the task was on a bad node, it could still be
rescheduled up to max.task.attempt times.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4085) Kill task attempts longer than a configured queue max time
[ https://issues.apache.org/jira/browse/MAPREDUCE-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-4085: Attachment: MAPREDUCE-4085-branch-1.0.txt Here's a code dump of what I'm currently working with on my test setup. I have a slightly different version running in front of users that is working well, but lacks the RPC support this one provides. Many thanks to those of you who have given some guidance on this... Kill task attempts longer than a configured queue max time -- Key: MAPREDUCE-4085 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4085 Project: Hadoop Map/Reduce Issue Type: New Feature Components: task Reporter: Allen Wittenauer Attachments: MAPREDUCE-4085-branch-1.0.txt For some environments, it is desirable to have certain queues have an SLA with regards to task turnover. (i.e., a slot will be free in X minutes and scheduled to the appropriate job) Queues should have a 'task time limit' that would cause task attempts over this time to be killed. This leaves open the possibility that if the task was on a bad node, it could still be rescheduled up to max.task.attempt times. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
