peter-toth commented on a change in pull request #31913:
URL: https://github.com/apache/spark/pull/31913#discussion_r598382819
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -870,8 +870,19 @@ object CollapseProject extends Rule[LogicalPlan] with
AliasHelper {
if (haveCommonNonDeterministicOutput(p.projectList,
agg.aggregateExpressions)) {
p
} else {
- agg.copy(aggregateExpressions = buildCleanedProjectList(
- p.projectList, agg.aggregateExpressions))
+ val complexGroupingExpressions =
+ ExpressionSet(agg.groupingExpressions.filter(_.children.nonEmpty))
+
+ def wrapGroupingExpression(e: Expression): Expression = e match {
+ case _: AggregateExpression => e
+ case _ if complexGroupingExpressions.contains(e) =>
GroupingExpression(e)
+ case _ => e.mapChildren(wrapGroupingExpression)
+ }
+
+ val wrappedAggregateExpressions =
+
agg.aggregateExpressions.map(wrapGroupingExpression(_).asInstanceOf[NamedExpression])
+ agg.copy(aggregateExpressions =
+ buildCleanedProjectList(p.projectList, wrappedAggregateExpressions))
Review comment:
Ahh you are right, even the initial query can contain terms in which
Spark should keep grouping expressions. I think I just inserted the
`GroupingExpression` wrapper around `id IS NULL` at the wrong place, maybe it
should happen during analysis. Will look into this today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
