[
https://issues.apache.org/jira/browse/HCATALOG-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439652#comment-13439652
]
Sushanth Sowmyan commented on HCATALOG-425:
-------------------------------------------
I've been thinking this through, and I agree with you. If we had datatype
conversions/promotions to make, it does make sense to be consistent about it at
the HCatRecord layer as opposed to in all tools/adapters above HCat. And my
primary concern is not that the datatype conversion is happening, but that
HCatSchema/HCatRecord contain differing data types.
Thus, one potential solution I have is as follows:
a) Make sure that the typed getters and setters in HCatRecord
(getInteger/setByte/etc) are aware of the fact that this translation was done
and that HCatRecord has a different type associated with it.
b) Change HCatFieldSchema.getType to return the "changed" type and maybe add
another function getOriginalType() or something to that effect that returns the
underlying type. That way, simple data readers/writers that work off
HCatRecord/HCatSchema will always be in synch, and conversions will be
invisible to them, while leaving the original info available for tools that
want more info.
c) In addition, if we find ourselves adding more conversion strategies beyond
the simple byte/short/boolean conversions to int, it might be worth looking to
refactor that by adding a common conf parameter that describes all conversions
taking place, and to load a "conversion strategy impl" if needed. (Personally,
I hope not, I think these make sense, but we shouldn't be adding too many
custom rules, or we risk slowing things down too much)
Any thoughts?
> Pig cannot read/write SMALLINT/TINYINT columns
> ----------------------------------------------
>
> Key: HCATALOG-425
> URL: https://issues.apache.org/jira/browse/HCATALOG-425
> Project: HCatalog
> Issue Type: Bug
> Components: pig
> Affects Versions: 0.4
> Reporter: Thejas M Nair
> Assignee: Travis Crawford
> Fix For: 0.5
>
> Attachments: HCATALOG-425_small_tiny_int.1.patch,
> HCATALOG-425_small_tiny_int.2.patch, HCATALOG-425_small_tiny_int.3.patch
>
>
> Currently throw exception. We can always allow read and on write side, we can
> do out of boundary check at runtime.
> This issue described in HCATALOG-168, has not been fixed. It still throws an
> exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
