[
https://issues.apache.org/jira/browse/HADOOP-1440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506877
]
Hudson commented on HADOOP-1440:
--------------------------------
Integrated in Hadoop-Nightly #131 (See
[http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/131/])
> JobClient should not sort input-splits
> --------------------------------------
>
> Key: HADOOP-1440
> URL: https://issues.apache.org/jira/browse/HADOOP-1440
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.12.3
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Senthil Subramanian
> Fix For: 0.14.0
>
> Attachments: HADOOP-1440_1.patch
>
>
> Currently, the JobClient sorts the InputSplits returned by InputFormat in
> descending order, so that the map tasks corresponding to larger input-splits
> are scheduled first for execution than smaller ones. However, this causes
> problems in applications that produce data-sets partitioned similarly to the
> input partition with -reducer NONE.
> With -reducer NONE, map task i produces part-i. Howver, in the typical
> applications that use -reducer NONE it should produce a partition that has
> the same index as the input parrtition.
> (Of course, this requires that each partition should be fed in its entirety
> to a map, rather than splitting it into blocks, but that is a separate issue.)
> Thus, sorting input splits should be either controllable via a configuration
> variable, or the FileInputFormat should sort the splits and JobClient should
> honor the order of splits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
