[
https://issues.apache.org/jira/browse/MAPREDUCE-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806042#action_12806042
]
Philip Zeyliger commented on MAPREDUCE-1126:
--------------------------------------------
@Scott: the annotations for Input/OutputFormat seem to be misplaced. It seems
desirable to be able to write a single Map function that does wordcount on
Strings, regardless of whether those strings are stored in newline-delimited
text, sequence files, avro data files, or whatever.
@Chris: "1) throwing away all Java type hierarchies". Only sometimes, no?
This is only in the case where you explicitly want to do unions (and Java's
union support is either Object, type hierarchies, or wrappers). In the typical
case, your map functions on SomeSpecificRecordType, outputs
SomeSpecificMapOutputKey/ValueType, and so forth. You still get type safety in
many of the recommended use cases.
> shuffle should use serialization to get comparator
> --------------------------------------------------
>
> Key: MAPREDUCE-1126
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1126
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Reporter: Doug Cutting
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1126.2.patch, MAPREDUCE-1126.3.patch,
> MAPREDUCE-1126.4.patch, MAPREDUCE-1126.5.patch, MAPREDUCE-1126.6.patch,
> MAPREDUCE-1126.patch, MAPREDUCE-1126.patch
>
>
> Currently the key comparator is defined as a Java class. Instead we should
> use the Serialization API to create key comparators. This would permit,
> e.g., Avro-based comparators to be used, permitting efficient sorting of
> complex data types without having to write a RawComparator in Java.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
