[
https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534087
]
Owen O'Malley commented on HADOOP-1986:
---------------------------------------
{quote}
Actually, I thought we might, and would like to preserve that option.
{quote}
WIth my proposal, of course that is an option since you could define a
Serializer class for each specific class. I don't see the value of it, but it
wouldn't be hard.
{quote}
I worry about performance of introspection.
{quote}
That does worry me too. I think that if you are just doing the look up once,
and caching the result, it won't be a problem.
{quote}
And, for very simple objects, the overhead of having
WritableSerializer#serialize(o,out) call o.write(out) rather than just being
o.write(out) may even be significant.
{quote}
*Laugh* I can't see it ever being anything but minor noise. JVMs are really
good at making method calls cheap.
{quote}
Or it may not be. In any case, if record code is generated from a DDL, then we
can implement this either way, with per-class serializers or per-baseclass
serializers. If we discard the DDL and code-generation, then we're stuck with
introspection, no?
{quote}
I assume that each serialization framework generates methods that convert
objects to bytes and visa versa. The problem is just that the interfaces aren't
the same. So the hadoop serializers never have to worry about DDLs, they just
worry about using each serialization framework's apis to do the conversion.
{quote}
I wonder if we might permit both by having the configuration name not
serializers but serializer factories.
{quote}
I think that 2 levels of indirection are overkill, but I'm open to discussing
it.
> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>
> Key: HADOOP-1986
> URL: https://issues.apache.org/jira/browse/HADOOP-1986
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 0.16.0
>
> Attachments: SerializableWritable.java
>
>
> Currently Map Reduce programs have to use WritableComparable-Writable
> key-value pairs. While it's possible to write Writable wrappers for other
> serialization frameworks (such as Thrift), this is not very convenient: it
> would be nicer to be able to use arbitrary types directly, without explicit
> wrapping and unwrapping.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
