Prasanth Jayachandran created HIVE-14324:
--------------------------------------------
Summary: ORC PPD for floats is broken
Key: HIVE-14324
URL: https://issues.apache.org/jira/browse/HIVE-14324
Project: Hive
Issue Type: Bug
Components: ORC
Affects Versions: 2.1.0, 2.0.0, 1.3.0, 2.2.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran
Priority: Critical
ORC stores min/max stats, bloom filters by passing floats as doubles using
java's widening conversion. So if we write a float value of 0.22 to ORC file,
the min/max stats and bloom filter will use 0.2199999988079071 double value.
But when we do PPD, SARG creates literals by converting float to string and
then to double which compares 0.22 to 0.2199999988079071 and fails PPD
evaluation.
{code}
hive> create table orc_float (f float) stored as orc;
hive> insert into table orc_float values(0.22);
hive> set hive.optimize.index.filter=true;
hive> select * from orc_float where f=0.22;
OK
hive> set hive.optimize.index.filter=false;
hive> select * from orc_float where f=0.22;
OK
0.22
{code}
This is not a problem for doubles and decimals.
This issue was introduced in HIVE-8460 but back then there was no strict type
check when SARGs are created and also PPD evaluation does not convert to column
type. But now predicate leaf creation in SARG enforces strict type check for
boxed literals and predicate type and PPD evaluation converts stats and
constants to column type (predicate).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
