[
https://issues.apache.org/jira/browse/MAPREDUCE-5758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003852#comment-14003852
]
Chen He commented on MAPREDUCE-5758:
------------------------------------
There are several issues we need to consider if we allow reducer use container
local directory
1) The MapReduce framework should get container local dir from YARN.
2) We need to let Yarn framework know that MapReduce framework created some
dirs under container local dir for reducers.
Any suggestion, [~vinodkv]?
> Reducer local data is not deleted until job completes
> -----------------------------------------------------
>
> Key: MAPREDUCE-5758
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5758
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.10, 2.2.0
> Reporter: Jason Lowe
> Assignee: Chen He
>
> Ran into an instance where a reducer shuffled a large amount of data and
> subsequently failed, but the local data is not purged when the task fails but
> only after the entire job completes. This wastes disk space unnecessarily
> since the data is no longer relevant after the task-attempt exits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
