[
https://issues.apache.org/jira/browse/MAPREDUCE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12752048#action_12752048
]
Ravi Gummadi commented on MAPREDUCE-956:
----------------------------------------
We could say the phases as Shuffle phase and Reduce phase. But we need to
investigate how we want to update progress in shuffle phase --- because
updating progress of shuffle phase just based on 'copy of map outputs' would
not be a correct way as there could be some merges that would take some time
after all map outputs are copied to this reduce node(even though some merges
happen while some map outputs are being copied).
> Shuffle should be broken down to only two phases (copy/reduce) instead of
> three (copy/sort/reduce)
> --------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-956
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-956
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
> Affects Versions: 0.21.0
> Reporter: Jothi Padmanabhan
>
> For the progress calculations and displaying on the UI, shuffle, in its
> current form, is decomposed into three phases (copy/sort/reduce). Actually,
> the sort phase is no longer applicable. I think we should just reduce the
> number of phases to two and assign 50% weight-age to each of copy and reduce
> phases. Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
