[
https://issues.apache.org/jira/browse/MAPREDUCE-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772007#action_12772007
]
Pradeep Kamath commented on MAPREDUCE-1151:
-------------------------------------------
Taking the cleanup task for discussion - conceptually this task runs at the end
of job run and the OutputCommitter implementation does not anticipate
setUpTask() , needsTaskCommit(), commitTask() and abortTask() methods being
called at the end of the job run -setupTask() for example might have logic
needed to setup the output for writing in a real map or reduce task - calling
them may have unwanted side effects like (say) temp directories being created
on hdfs for setup. To keep semantics right, it would seem that the cleanup task
should only call the cleanupJob() method and setup task should only call
setupJob() for the same reasons of not causing unwanted side effects - thoughts?
> Cleanup and Setup jobs should only call cleanupJob() and setupJob() methods
> of the OutputCommitter
> --------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1151
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1151
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Pradeep Kamath
>
> The cleanup and setup jobs run as map jobs and call setUpTask() ,
> needsTaskCommit() and possibly commitTask() and abortTask() methods of the
> OutputCommitter. They should only be calling the cleanupJob() and setupJob()
> methods.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
