[
https://issues.apache.org/jira/browse/HIVE-4736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Hanson updated HIVE-4736:
------------------------------
Summary: Float aggregate of single value loses precission (was: Float
aggregate of single value looses precission)
> Float aggregate of single value loses precission
> ------------------------------------------------
>
> Key: HIVE-4736
> URL: https://issues.apache.org/jira/browse/HIVE-4736
> Project: Hive
> Issue Type: Sub-task
> Components: Query Processor
> Affects Versions: vectorization-branch
> Reporter: Remus Rusanu
> Assignee: Remus Rusanu
> Priority: Minor
>
> I am seeing differences that are small but greater than expected in the
> output of TPC-H query 1. Thanks to Jitendra’s patch, the negative result
> error is gone. However, I am seeing that the results for
>
> Sum(l_extendedprice * ( 1 - l_discount ))
> Sum(l_extendedprice * ( 1 - l_discount ) * ( 1 + l_tax ))
>
> Differ in the low-order 8 or so digits compared with text. I know the input
> is float so it should have all zeros in the low digits when cast to double. I
> would not expect the answers to match exactly but I would expect errors in
> the last few digits, not the last 8. I could be wrong, but I think this may
> be worth investigating. Any ideas?
>
> I also noticed this, where I ran the query against a single row worth of
> input (order 1, line 1):
>
> SELECT l_returnflag,
> l_linestatus,
> Sum(l_quantity) AS sum_qty,
> Sum(l_extendedprice) AS
> sum_base_price,
> Sum(l_extendedprice * ( 1 - l_discount )) AS
> sum_disc_price,
> Sum(l_extendedprice * ( 1 - l_discount ) * ( 1 + l_tax )) AS
> sum_charge,
> Avg(l_quantity) AS avg_qty,
> Avg(l_extendedprice) AS avg_price,
> Avg(l_discount) AS avg_disc,
> Count(*) AS
> count_order
> FROM lineitem
> WHERE l_shipdate <= '1998-09-19'
> and l_orderkey = 1 and l_linenumber = 1
> GROUP BY l_returnflag,
> l_linestatus
> ORDER BY l_returnflag,
> l_linestatus;
>
> input row
> 1 155190 7706 1 17.0 21168.23 0.04 0.02 N
> O 1996-03-13 1996-02-12 1996-03-22 DELIVER IN PERSON TRUCK
> egular courts above the
>
>
> Result
>
> V (orc)
> N O 17.0 21168.23046875 20321.501268925873
> 20727.931285219973 17.0 21168.23046875 0.03999999910593033 1
>
> NV (text)
> N O 17.0 21168.23046875 20321.5 20727.9296875
> 17.0 21168.23046875 0.03999999910593033 1
--
This message was sent by Atlassian JIRA
(v6.1#6144)
