[
https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531954
]
Doug Cutting commented on HADOOP-1986:
--------------------------------------
> But if the serializer is specific to the job, you wouldn't be able to mix
> Writables and Thrift objects.
We need a serializer and deserializer specified per job so that the mapred
kernel can store intermediate data. Then the InputFormat may use a
deserializer, and the OutputFormat may use a serializer. So I don't see that
Tom's proposal (at least not as I interpret it) prohibits such intermixing.
The job's serializer only applies to the map output. The InputFormat's
deserializer would apply to the map input, and the OutputFormat's deserializer
would apply to reduce output. Does that make sense?
> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>
> Key: HADOOP-1986
> URL: https://issues.apache.org/jira/browse/HADOOP-1986
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Reporter: Tom White
> Fix For: 0.16.0
>
>
> Currently Map Reduce programs have to use WritableComparable-Writable
> key-value pairs. While it's possible to write Writable wrappers for other
> serialization frameworks (such as Thrift), this is not very convenient: it
> would be nicer to be able to use arbitrary types directly, without explicit
> wrapping and unwrapping.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
