[
https://issues.apache.org/jira/browse/HCATALOG-364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Travis Crawford updated HCATALOG-364:
-------------------------------------
Attachment: HCATALOG-364_pig_boolean_support.2.patch
Thanks for the review Ashutosh! I updated the comment and committed the
attached patch.
> HCatalog pig adapter should handle boolean fields
> -------------------------------------------------
>
> Key: HCATALOG-364
> URL: https://issues.apache.org/jira/browse/HCATALOG-364
> Project: HCatalog
> Issue Type: New Feature
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Fix For: 0.5
>
> Attachments: HCATALOG-364_pig_boolean_support.1.patch,
> HCATALOG-364_pig_boolean_support.2.patch
>
>
> Currently HCatalog does not support boolean fields, explicitly looking for
> them and throwing an exception:
> {code}
> Caused by: org.apache.pig.PigException: ERROR 1115: Incompatible type found
> in hcat table schema: boolean
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHcatFieldFollowsPigRules(PigHCatUtil.java:403)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatSchemaFollowsPigRules(PigHCatUtil.java:391)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHcatFieldFollowsPigRules(PigHCatUtil.java:408)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatSchemaFollowsPigRules(PigHCatUtil.java:391)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatTableSchemaFollowsPigRules(PigHCatUtil.java:422)
> at org.apache.hcatalog.pig.HCatLoader.getSchema(HCatLoader.java:158)
> {code}
> This is a big problem as many of our schemas use boolean.
> I propose we take the same approach as elephant-bird's
> [ThriftToPig.java|https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/pig/util/ThriftToPig.java#L106]
> and translate into an Integer in the pig layer. This approach allows
> processing frameworks with boolean support to take advantage of that feature,
> and frameworks without boolean support could use the translated value
> (versions of pig people are likely to have deployed; newer pig versions with
> boolean support could use the type directly). We've been using this technique
> for some time and in practice it works well.
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
