[
https://issues.apache.org/jira/browse/HIVE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733866#action_12733866
]
Yuntao Jia commented on HIVE-640:
---------------------------------
Changes need to be made to the source code:
1/ Create "LazyBinary" classes for all data types. It will be a hierarchy of
classes which is similar to the hierarchy of the "Lazy" classes.
The most top level class is LazyBinaryObject which has two child classes,
LazyBinaryPrimitive and LazyBinaryNonPrimitive.
LazyBinaryPrimitive has several child classes defined for all primitive
types, such as LazyBinaryBoolean for Boolean, LazyBinaryInteger
for Integer and so on. LazyBinaryNonPrimitive has 4 child classes for 4
complex data types in Hive, including LazyBinaryStruct,
LazyBinaryMap, LazyBinaryList and LazyBinaryString. The reason of having
LazyBinaryPrimitive and its child classes is that we
can easily define a LazyBinaryList as a list of LazyPrimitive objects.
2/ Create LazyBinary object inspectors for some of the LazyBinary classes that
handle complex data types. They are
LazyBinaryStructObjectInspector, LazyBinaryMapObjectInspector,
LazyBinaryListObjectInspector and LazyBinaryStringObjectInspector.
3/ Create the LazyBinarySerDe class which serializes a datastream to LazyBinary
format and later deserialize them back to LazyBinary types.
> Add LazyBinarySerDe to Hive
> ---------------------------
>
> Key: HIVE-640
> URL: https://issues.apache.org/jira/browse/HIVE-640
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Zheng Shao
> Assignee: Yuntao Jia
>
> LazyBinarySerDe will serialize the data in binary format while supporting
> LazyDeserialization.
> This will be used as the SerDe for value between map and reduce, and also
> between different map-reduce jobs.
> This will help improve the performance of Hive a lot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.