[
https://issues.apache.org/jira/browse/HIVE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170367#comment-17170367
]
Girish Kadli commented on HIVE-3562:
------------------------------------
I have a hive query its returning different results with and without limit.
Let's say with limit query result set as R1 and without limit query result set
as R2.
These are the following discrepancies:
* R1 contains some of the column values as null.
* R2 doesn't contain the rows returned by R1.
* R2 contains all non null column values.
* R2 is returning correct results, R1 is returning wrong results.
After debugging realised that *hive.limit.pushdown.memory.usage=0.1*
is the root cause of this issue. after i set this property to -1, R1 starts
returning correct rows with non null column values. and R1 results are part of
R2 results.
What could be the problem setting lower value to
*hive.limit.pushdown.memory.usage?*
can it cause data issues in "with limit" hive queries by returning wrong
results?
> Some limit can be pushed down to map stage
> ------------------------------------------
>
> Key: HIVE-3562
> URL: https://issues.apache.org/jira/browse/HIVE-3562
> Project: Hive
> Issue Type: Bug
> Reporter: Navis Ryu
> Assignee: Navis Ryu
> Priority: Trivial
> Fix For: 0.12.0
>
> Attachments: HIVE-3562.D5967.1.patch, HIVE-3562.D5967.2.patch,
> HIVE-3562.D5967.3.patch, HIVE-3562.D5967.4.patch, HIVE-3562.D5967.5.patch,
> HIVE-3562.D5967.6.patch, HIVE-3562.D5967.7.patch, HIVE-3562.D5967.8.patch,
> HIVE-3562.D5967.9.patch
>
>
> Queries with limit clause (with reasonable number), for example
> {noformat}
> select * from src order by key limit 10;
> {noformat}
> makes operator tree,
> TS-SEL-RS-EXT-LIMIT-FS
> But LIMIT can be partially calculated in RS, reducing size of shuffling.
> TS-SEL-RS(TOP-N)-EXT-LIMIT-FS
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
