[
https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931517#action_12931517
]
Tom White commented on HADOOP-6685:
-----------------------------------
> If one's records don't implement the {{Writable}} interface, then there's no
> reasonable binary container in Hadoop.
SequenceFile supports non-Writable types already. The limitation today is that
there must be a one-to-one mapping between Java class and serialized data type.
I think that can be satisfied by both Thrift and Protocol Buffers. For Avro, I
don't think we want to support it in SequenceFile, as we should instead
encourage use of Avro Data File, which is like SequenceFile but interoperable
with other languages.
A process question: given how we failed to gain consensus last time, what could
we do differently this time round? A design document to motivate the use cases?
Any other suggestions?
> Change the generic serialization framework API to use serialization-specific
> bytes instead of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6685
> URL: https://issues.apache.org/jira/browse/HADOOP-6685
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: serial.patch
>
>
> Currently, the generic serialization framework uses Map<String,String> for
> the serialization specific configuration. Since this data is really internal
> to the specific serialization, I think we should change it to be an opaque
> binary blob. This will simplify the interface for defining specific
> serializations for different contexts (MAPREDUCE-1462). It will also move us
> toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
