ulysses-you commented on code in PR #57276:
URL: https://github.com/apache/spark/pull/57276#discussion_r3592803820
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -2999,6 +2999,17 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val COMBINE_ADJACENT_AGGREGATION_ENABLED =
+ buildConf("spark.sql.execution.combineAdjacentAggregation")
+ .internal()
+ .doc("When true, combine adjacent aggregation with `Partial` and `Final`
to `Complete` " +
+ "mode. This defaults to the value of
`spark.sql.execution.replaceHashWithSortAgg` since " +
+ "combining adjacent aggregation subsumes the partial-and-final merge
that " +
+ "`replaceHashWithSortAgg` used to perform on its own.")
+ .version("4.3.0")
+ .withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
+ .fallbackConf(REPLACE_HASH_WITH_SORT_AGG_ENABLED)
Review Comment:
Before, the rule `ReplaceHashWithSortAgg` supports merge adjacent
aggregation. This pr pulls out related code and make a new rule
`CombineAdjacentAggregation`. So fallback to REPLACE_HASH_WITH_SORT_AGG_ENABLED
config makes behavior same as before.
I plan to enable both of these two rules in an another pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
