[
https://issues.apache.org/jira/browse/HIVE-24479?focusedWorklogId=519709&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-519709
]
ASF GitHub Bot logged work on HIVE-24479:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Dec/20 15:05
Start Date: 03/Dec/20 15:05
Worklog Time Spent: 10m
Work Description: kasakrisz opened a new pull request #1734:
URL: https://github.com/apache/hive/pull/1734
### What changes were proposed in this pull request?
Introduce a new HiveConf setting to set lower bound of hash aggregation
reduction. The default value is 0.5.
During query compilation hash aggregation reduction is adjusted by
calculating its effectiveness.
With this patch if the adjusted reduction value is less than the configured
lower bound the lower bound value will be used.
### Why are the changes needed?
Some cases we end up with 0 values forcing the Group by operator to skip
Hash aggregates and chooses streaming mode.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 519709)
Remaining Estimate: 0h
Time Spent: 10m
> Introduce setting to set lower bound of hash aggregation reduction.
> -------------------------------------------------------------------
>
> Key: HIVE-24479
> URL: https://issues.apache.org/jira/browse/HIVE-24479
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
> Affects Versions: 4.0.0
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Fix For: 4.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> * Default setting of hash group by min reduction % is 0.99.
> * During compilation, we check its effectiveness and adjust it accordingly in
> {{SetHashGroupByMinReduction}}:
> {code}
> float defaultMinReductionHashAggrFactor = desc.getMinReductionHashAggr();
> float minReductionHashAggrFactor = 1f - ((float) ndvProduct / numRows);
> if (minReductionHashAggrFactor < defaultMinReductionHashAggrFactor) {
> desc.setMinReductionHashAggr(minReductionHashAggrFactor);
> }
> {code}
> For certain queries, this computation turns out to be "0".
> This forces operator to skip HashAggregates completely and always ends up
> choosing streaming mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
