[
https://issues.apache.org/jira/browse/MAPREDUCE-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831013#action_12831013
]
Doug Cutting commented on MAPREDUCE-1126:
-----------------------------------------
> Tom's patch adds two calls to that exact method.
One call is in a test method called getWritableSerializationMap, clearly
serializaton-specific code. The other is in reading input splits, which has
not been converted to use the new serialization API, and still uses the split's
class alone to determine serialization.
A primary goal of this issue is to update the shuffle to use the new
Map<String,String> means of naming serializations, so that any serialzation may
be used, not just those determinable from the class of the instance to be
serialized. Your patch uses the map output key class alone to determine the
serialization, and hence does not meet this goal.
> shuffle should use serialization to get comparator
> --------------------------------------------------
>
> Key: MAPREDUCE-1126
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1126
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Reporter: Doug Cutting
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: m-1126-2.patch, m-1126-3.patch, MAPREDUCE-1126.2.patch,
> MAPREDUCE-1126.3.patch, MAPREDUCE-1126.4.patch, MAPREDUCE-1126.5.patch,
> MAPREDUCE-1126.6.patch, MAPREDUCE-1126.patch, MAPREDUCE-1126.patch
>
>
> Currently the key comparator is defined as a Java class. Instead we should
> use the Serialization API to create key comparators. This would permit,
> e.g., Avro-based comparators to be used, permitting efficient sorting of
> complex data types without having to write a RawComparator in Java.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
