rubenada commented on code in PR #6963:
URL: https://github.com/apache/hive/pull/6963#discussion_r4587528119
##########
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##########
@@ -1814,6 +1814,27 @@ public static enum ConfVars {
AGGR_JOIN_TRANSPOSE("hive.transpose.aggr.join", false, "push aggregates
through join"),
AGGR_JOIN_TRANSPOSE_UNIQUE("hive.transpose.aggr.join.unique", true, "push
aggregates through join(s) in "
+ "case data is regrouped on a previously unique column"),
+ PARTIAL_AGGR_JOIN_TRANSPOSE("hive.optimize.partial.aggr.join.transpose",
true,
+ "Whether to push a map-side partial aggregate below an inner join, to
the join input that holds all\n"
+ + "aggregate arguments. The pushed aggregate runs in the vertex of the
join input, before its existing\n"
+ + "shuffle. The map-side aggregate above the join stays and merges the
partial results before the shuffle\n"
+ + "of the aggregate."),
+
PARTIAL_AGGR_JOIN_TRANSPOSE_MIN_REDUCTION("hive.optimize.partial.aggr.join.transpose.min.reduction",
0.8f,
+ "Hash aggregation of a pushed partial aggregate is turned off when the
ratio between hash table size and\n"
+ + "input rows is bigger than this number, and its rows then pass
through as partial results."
+ + " hive.map.aggr.hash.min.reduction stays the upper bound, and with\n"
+ + "hive.map.aggr.hash.min.reduction.stats the statistics may lower it
further."),
+
PARTIAL_AGGR_JOIN_TRANSPOSE_MAX_JOIN_REDUCTION("hive.optimize.partial.aggr.join.transpose.max.join.reduction",
10f,
+ "A partial aggregate is not pushed below a join estimated to reduce
the pushed input by more than this\n"
+ + "factor: a map join has no dynamic filter on its probe side, so the
pushed aggregate would process every\n"
+ + "row the join discards. 0 disables the check."),
+
PARTIAL_AGGR_JOIN_TRANSPOSE_MAX_JOIN_EXPANSION("hive.optimize.partial.aggr.join.transpose.max.join.expansion",
+ 1.1f,
+ "A partial aggregate is not pushed below a join estimated to output
more rows than this factor times the\n"
Review Comment:
I'm not sure I understand this, why do we need a MAX_JOIN_EXPANSION? IIUC an
expanding join is where aggregating early should help most, or am I missing
something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]