[
https://issues.apache.org/jira/browse/HIVE-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673776#action_12673776
]
Joydeep Sen Sarma commented on HIVE-270:
----------------------------------------
a couple of things i missed in first review:
- where we get numberformatexception in lazy number parsing - should we return
serdeexception instead? I am just worried that by sending them across as nulls
- we are totally hiding any errors from the user.
right now the serdeexceptions just go into a hadoop counter - but perhaps
later we can surface them to the user (and error out job on threshold number of
exceptions)
- can we replace all occurences of metadatatypedserde? it seems that there are
quite a few places in the code that are still using it (see
planutils.getdefaulttabledesc for instance - that goes into a bunch of places -
like script operator, fetch task, filesink)
> Add a lazy-deserialized SerDe for space and cpu efficient serialization of
> rows with primitive types
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-270
> URL: https://issues.apache.org/jira/browse/HIVE-270
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-270.1.patch, HIVE-270.3.patch, HIVE-270.4.patch,
> HIVE-270.5.patch
>
>
> We want to add a lazy-deserialized SerDe for space and cpu efficient
> serialization of rows with primitive types.
> This SerDe will share the same format as
> MetadataTypedColumnsetSerDe/TCTLSeparatedProtocol to be backward compatible.
> This SerDe will be used to replace the default table SerDe, and the SerDe
> used to communicate with user scripts.
> For simplicity, we don't plan to support nested structure with this SerDe.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
