[
https://issues.apache.org/jira/browse/HADOOP-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Gummadi updated HADOOP-5210:
---------------------------------
Attachment: HADOOP-5210.v2.patch
Yes Jothi. When the intermediate merges complete, we can say that the sortPhase
is completed and if we reset the variable totalBytesProcessed before the final
merge, we can use that for calculating the progress of reducePhase(the 3rd
phase of reduce task). Patch of HADOOP-3131 removed this resetting of
totalBytesProcessed.
Matei, Would you please check if your patch(of JIRA 3131) removed this reset
intentionally and if I am missing out something ?
Attaching patch which resets the bytes-processed to zero before final merge.
Please review and provide your comments.
> Reduce Task Progress shows > 100% when the total size of map outputs (for a
> single reducer) is high
> ----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5210
> URL: https://issues.apache.org/jira/browse/HADOOP-5210
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Jothi Padmanabhan
> Assignee: Ravi Gummadi
> Priority: Minor
> Attachments: HADOOP-5210.patch, HADOOP-5210.v2.patch, Picture 3.png
>
>
> When the total map outputs size (reduce input size) is high, the reported
> progress is greater than 100%.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
