[ https://issues.apache.org/jira/browse/HIVE-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Remus Rusanu updated HIVE-6873: ------------------------------- Attachment: image004.png image003.png image002.png image001.png set hive.map.groupby.sorted=true; was the missing part for me… From: Jitendra Nath Pandey (JIRA) [mailto:j...@apache.org] Sent: Friday, April 11, 2014 1:02 AM To: Remus Rusanu Subject: [jira] [Commented] (HIVE-6873) DISTINCT clause in aggregates is handled incorrectly by vectorized execution [cid:image001.png@01CF557F.FDF58F90] Jitendra Nath Pandey<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jnp> commented on [Bug] HIVE-6873<https://issues.apache.org/jira/browse/HIVE-6873> Re: DISTINCT clause in aggregates is handled incorrectly by vectorized execution<https://issues.apache.org/jira/browse/HIVE-6873> Here is a scenario where we get incorrect result. It shows up on sorted bucketed column with hive.map.groupby.sorted=true, and only on group by queries with no keys. Here are the steps: hive> Create table T(a int, b int) clustered by (a) sorted by (a) stored as orc; load following data: 300 1 300 1 300 1 300 1 300 1 hive> set hive.map.groupby.sorted=true; hive> select sum(distinct a) from T; // Incorrect result. hive> select count(distinct a) from T; // This is also incorrect. [Add Comment]<https://issues.apache.org/jira/browse/HIVE-6873#add-comment> Add Comment<https://issues.apache.org/jira/browse/HIVE-6873#add-comment> This message was sent by Atlassian JIRA (v6.2#6252-sha1:aa34325) [Atlassian logo] > DISTINCT clause in aggregates is handled incorrectly by vectorized execution > ---------------------------------------------------------------------------- > > Key: HIVE-6873 > URL: https://issues.apache.org/jira/browse/HIVE-6873 > Project: Hive > Issue Type: Bug > Components: Query Processor > Affects Versions: 0.13.0, 0.14.0 > Reporter: Remus Rusanu > Assignee: Remus Rusanu > Attachments: HIVE-6873.1.patch, HIVE-6873.2.patch, HIVE-6873.3.patch, > image001.png, image002.png, image003.png, image004.png > > > The vectorized aggregates ignore the DISTINCT clause. This cause incorrect > results. Due to how GroupByOperatorDesc adds the DISTINCT keys to the overall > aggregate keys the vectorized aggregates do account for the extra key, but > they do not process the data correctly for the key. the reduce side the > aggregates the input from the vectorized map side to results that are only > sometimes correct but mostly incorrect. HIVE-4607 tracks the proper fix, but > meantime I'm filing a bug to disable vectorized execution if DISTINCT is > present. Fix is trivial. -- This message was sent by Atlassian JIRA (v6.2#6252)