rubenada commented on code in PR #6963:
URL: https://github.com/apache/hive/pull/6963#discussion_r4587528119


##########
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##########
@@ -1814,6 +1814,27 @@ public static enum ConfVars {
     AGGR_JOIN_TRANSPOSE("hive.transpose.aggr.join", false, "push aggregates 
through join"),
     AGGR_JOIN_TRANSPOSE_UNIQUE("hive.transpose.aggr.join.unique", true, "push 
aggregates through join(s) in "
         + "case data is regrouped on a previously unique column"),
+    PARTIAL_AGGR_JOIN_TRANSPOSE("hive.optimize.partial.aggr.join.transpose", 
true,
+        "Whether to push a map-side partial aggregate below an inner join, to 
the join input that holds all\n"
+        + "aggregate arguments. The pushed aggregate runs in the vertex of the 
join input, before its existing\n"
+        + "shuffle. The map-side aggregate above the join stays and merges the 
partial results before the shuffle\n"
+        + "of the aggregate."),
+    
PARTIAL_AGGR_JOIN_TRANSPOSE_MIN_REDUCTION("hive.optimize.partial.aggr.join.transpose.min.reduction",
 0.8f,
+        "Hash aggregation of a pushed partial aggregate is turned off when the 
ratio between hash table size and\n"
+        + "input rows is bigger than this number, and its rows then pass 
through as partial results."
+        + " hive.map.aggr.hash.min.reduction stays the upper bound, and with\n"
+        + "hive.map.aggr.hash.min.reduction.stats the statistics may lower it 
further."),
+    
PARTIAL_AGGR_JOIN_TRANSPOSE_MAX_JOIN_REDUCTION("hive.optimize.partial.aggr.join.transpose.max.join.reduction",
 10f,
+        "A partial aggregate is not pushed below a join estimated to reduce 
the pushed input by more than this\n"
+        + "factor: a map join has no dynamic filter on its probe side, so the 
pushed aggregate would process every\n"
+        + "row the join discards. 0 disables the check."),
+    
PARTIAL_AGGR_JOIN_TRANSPOSE_MAX_JOIN_EXPANSION("hive.optimize.partial.aggr.join.transpose.max.join.expansion",
+        1.1f,
+        "A partial aggregate is not pushed below a join estimated to output 
more rows than this factor times the\n"

Review Comment:
   I'm not sure I understand this, why do we need a MAX_JOIN_EXPANSION? IIUC an 
expanding join is where aggregating early should help most, or am I missing 
something?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to