[
https://issues.apache.org/jira/browse/HIVE-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sushanth Sowmyan updated HIVE-4816:
-----------------------------------
Attachment: 4816.patch
Attaching the original exploratory patch I had up on HIVE-4551 that introduces
notion of a DataTransformer and a TransformedHCatRecord.
This patch still has one important bug that needs to be ironed out - I attach
it mostly to show what I intended by my previous comment.
The bug is that it still doesn't do conversion of shorts/bytes inside a nested
structure, and so even if all the tests currently pass with this patch, this
patch is broken(an indication that we need more tests ). Essentially, a lot
more of PigHCatUtil.getJavaObj needs to move into the DataTransformer, and out
of PigHCatUtil.
> Followup from HIVE-4551 : Reworking HCat Record Transformation
> --------------------------------------------------------------
>
> Key: HIVE-4816
> URL: https://issues.apache.org/jira/browse/HIVE-4816
> Project: Hive
> Issue Type: Improvement
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: 4816.patch
>
>
> This is a post-bug followup from HIVE-4551, to refactor some HCatalog code to
> be cleaner/neater. Reposting contents from my previous jira, this jira needs
> to track the following:
> a) HCatRecord and HCatSchema reflect underlying raw data and do no promotions.
> b) Introduce a ConversionImpl, which defines various datatype conversion
> functions, which all default to returning the input, and having a config that
> allows a user which conversions are implemented.
> c) Introduce a PromotedHCatRecord & PromotedHCatSchema that wrap
> HCatRecord/HCatSchema and use a ConversionImpl.
> d) Implement a PigLoaderConversionImpl/PigStorerConversionImpl in
> hcat-pig-adapter, which implements the following: Short->Int promotion,
> Short->Int promotion, Boolean->Int promotion
> e) Have HCatLoader/HCatStorer use the promoted versions of
> HCatRecord/HCatSchema which use the PigConversionImpl.
> f) Remove the current HCatContext promotion parameters and make them be
> HCatLoader/HCatStorer parameters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
