[
https://issues.apache.org/jira/browse/HADOOP-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12521602
]
Andrzej Bialecki commented on HADOOP-1727:
-------------------------------------------
I spotted another potential problem, which indirectly comes from the removal of
static methods and fields (which I agree with).
The addToMap method should allow one to explicitly set the id, and the
initialization of the maps in the AbstractMapWritable constructor should use
pre-defined id-s (they could start from -127 for that matter, while
automatically added mappings would start from 0 up to 127 - then we could avoid
using getKnownClasses(), because all negative id-s would be pre-defined by
definition).
The reason for this proposed change is the future backward compatibility, and
extensibility.
First, the compatibility. Let's assume that one day we deprecate ObjectWritable
in favor of GenericWritable, and then in the next release we remove
ObjectWritable completely ... oops, we can no longer read old data, even if we
didn't use ObjectWritable at all, because some of the remaining class->id
mappings are now shifted by one. Therefore, the mappings should be fixed for
all remaining classes across releases, and new classes should not re-use old
id-s for a long time.
I propose the following change then:
protected addToMap(Class clazz, byte id);
protected addToMap(Class clazz);
The second method would automatically assign id-s from 0 up to 127 for
arbitrary user types, the first method would be used just to register
pre-defined id-s.
Second, the extensibility. In Nutch we will want to subclass MapWritable, and
register additional frequently used types, so that they use pre-defined id-s
instead of the serialized mapping, for compactness. With the current
implementation it's not possible to do this.
> Make ...hbase.io.MapWritable more generic so that it can be included in
> ...hadoop.io
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-1727
> URL: https://issues.apache.org/jira/browse/HADOOP-1727
> Project: Hadoop
> Issue Type: New Feature
> Components: io
> Affects Versions: 0.15.0
> Reporter: Jim Kellerman
> Assignee: Jim Kellerman
> Priority: Minor
> Fix For: 0.15.0
>
> Attachments: patch.txt, patch.txt, patch.txt, patch.txt, patch.txt
>
>
> The class org.apache.hadoop.hbase.io.MapWritable could be made more generic
> through the use of ReflectionUtils so that it could support more Map key and
> value classes. Currently it supports Map<WritableComparable, Writable> only.
> When more generalized, submit for consideration to be included in
> org.apache.hadoop.io
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
