[
https://issues.apache.org/jira/browse/MAPREDUCE-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799448#action_12799448
]
Todd Lipcon commented on MAPREDUCE-1367:
----------------------------------------
- MapTaskRunnable: make most of the instance variables final?
- it should be clearer that mapOutputFiles is an "out param" for
getMapTaskRunnables - ie that it's expected to be intitially empty.
Alternatively, it looks like you may be able to remove this variable entirely
by instead storing the MapOutputFile in the MapRunnable instance, and then
iterating directly over the MapRunnables in the reducer. Does that make sense?
However, the way you've done it is less invasive to the reduce side, so if you
don't see the benefit, feel free to ignore this suggestion.
- Does this handle the degenerate case of 0-map jobs? It sounds ridiculous, but
I recall previous JIRAs for this situation, since occasionally people have a
cron job that periodically processes a given directory. If the directory is
empty, it may generate a job with no input splits and thus no tasks.
Aside from that, looks good to me.
> LocalJobRunner should support parallel mapper execution
> -------------------------------------------------------
>
> Key: MAPREDUCE-1367
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1367
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1367.2.patch, MAPREDUCE-1367.3.patch,
> MAPREDUCE-1367.4.patch, MAPREDUCE-1367.patch
>
>
> The LocalJobRunner currently supports only a single execution thread. Given
> the prevalence of multi-core CPUs, it makes sense to allow users to run
> multiple tasks in parallel for improved performance on small (local-only)
> jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
