-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7432/
-----------------------------------------------------------
Review request for hcatalog.
Description
-------
Currently the HCatalog pig adapter explicitly disallows boolean fields. This
made sense at the time it was written because no released version of Pig
supported boolean fields. However, starting with 0.10.0 Pig does have boolean
support.
This patch checks if the Pig version on the classpath supports boolean fields,
and if so, enables support. Users of pre-boolean support pig versions are
presented with a log line cluing them into the feature that converts boolean
fields to ints (hcat.data.convert.boolean.to.integer=true). Without setting the
property pig will error out due to an unsupported field type.
This addresses bug HCATALOG-364.
https://issues.apache.org/jira/browse/HCATALOG-364
Diffs
-----
hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/PigHCatUtil.java
2407de5
Diff: https://reviews.apache.org/r/7432/diff/
Testing
-------
Tests pass:
https://travis.ci.cloudbees.com/job/HCATALOG-364_pig_boolean_support/
I also tested with pig 0.9 (with hcat.data.convert.boolean.to.integer=true) and
0.11 with the following script:
"""
register
/home/travis/tmp/hcatalog/share/hcatalog/hcatalog-{core,pig-adapter}-0.5.0-SNAPSHOT.jar;
register /usr/lib/hive/lib/*.jar;
raw_data = load 'mytable' using org.apache.hcatalog.pig.HCatLoader();
data = filter raw_data by part_dt == '20121003T000000Z';
describe data;
a = foreach data generate myboolfield;
b = group a by myboolfield;
c = foreach b generate group, COUNT(a.myboolfield);
dump c;
"""
pig 0.11:
(true,4325)
(false,23093032)
pig 0.9:
(0,23093032)
(1,4325)
Thanks,
Travis Crawford
