[ https://issues.apache.org/jira/browse/MAPREDUCE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435690#comment-15435690 ]
Haibo Chen commented on MAPREDUCE-6501: --------------------------------------- Task 2 is already implemented with taskAttemptProgress, that is, SHUFFLE phase is expressed in terms of progress, and preemption also tries to preempt reducers that are of least progress. Task 3 is inapplicable since reducer preemption never kicks in if there is one mapper can run. > Improve reducer preemption > -------------------------- > > Key: MAPREDUCE-6501 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6501 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster > Affects Versions: 2.7.1 > Reporter: Karthik Kambatla > Assignee: Haibo Chen > > As discussed on MAPREDUCE-6302, we could improve the reducer preemption as > follows: > # preempt enough reducers so there are enough mappers to match slowstart > # prioritize preempting reducers that are still in SHUFFLE phase > # add an option to not preempt reducers that are past SHUFFLE phase > irrespective of slowstart as long as one mapper can run > We could do it all in one patch or create subtasks as necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org