[
https://issues.apache.org/jira/browse/MAPREDUCE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036751#comment-13036751
]
Devaraj K commented on MAPREDUCE-2512:
--------------------------------------
This try/catch is not related to the any stuck tasks. If there are no map
outputs ready to copy, this thread goes to sleep state for 5000 millis and then
continue execution after timeout. When this thread is in sleep state, if any
thread performs some operation (like interrupt (), stop ()) on this thread, it
will throw InterruptedException and comes out from the sleep state. After
getting InterruptedException, it will ignore and continue execution normally.
This thread waits for 5000 millis even If it gets new map completion events
before 5000 millis. This can be optimized such that,
{code:title=ReduceTask.java|borderStyle=solid}
reporter.progress();
- Thread.sleep(5000);
+ synchronized (lockCopy) {
+ lockCopy.wait(5000);
+ }
}
} catch (InterruptedException e) { } // IGNORE
int numNewMaps = getMapCompletionEvents();
if (numNewMaps > 0) {
+ synchronized (lockCopy) {
+ lockCopy.notify();
+ }
LOG.info(reduceTask.getTaskID() + ": " +
"Got " + numNewMaps + " new map-outputs");
{code}
> wait(5000) and notify() mechanism can be implemented instead of sleep(5000)
> in reduce task when there are no copies in progress and no new copies to
> schedule
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2512
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2512
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 0.20.2
> Reporter: Devaraj K
> Assignee: Devaraj K
>
> {code:title=ReduceTask.java|borderStyle=solid}
> try {
> if (numInFlight == 0 && numScheduled == 0) {
> // we should indicate progress as we don't want TT to think
> // we're stuck and kill us
> reporter.progress();
> Thread.sleep(5000);
> }
> } catch (InterruptedException e) { } // IGNORE
> {code}
> Here if we have no copies in flight and we can't schedule anything new, it is
> going to wait for 5000 millis. Instead of waiting for 5000 millis, this
> thread can wait with timeout and GetMapEventsThread can notify it if gets new
> map completion events earlier than 5000 millis time.
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
