Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/19175#discussion_r140717827
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1287,3 +1288,33 @@ object RemoveRepetitionFromGroupExpressions extends
Rule[LogicalPlan] {
a.copy(groupingExpressions = newGrouping)
}
}
+
+/**
+ * Splits [[Aggregate]] on [[Expand]], which has large number of
projections,
+ * into various [[Aggregate]]s.
+ */
+object SplitAggregateWithExpand extends Rule[LogicalPlan] {
+ /**
+ * Split [[Expand]] operator to a number of [[Expand]] operators
+ */
+ private def splitExpand(expand: Expand): Seq[Expand] = {
+ val len = expand.projections.length
+ val allProjections = expand.projections
+ Seq.tabulate(len)(
+ i => Expand(Seq(allProjections(i)), expand.output, expand.child)
--- End diff --
Why not remove the Expand altogether. The only thing you you might need to
do is to make sure that the expressions produce the correct attributes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
