uros-b commented on code in PR #58144:
URL: https://github.com/apache/spark/pull/58144#discussion_r3873010954
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala:
##########
@@ -1834,12 +1834,339 @@ object CollapseWindow extends Rule[LogicalPlan] {
}
/**
- * Transpose Adjacent Window Expressions.
- * - If the partition spec of the parent Window expression is compatible with
the partition spec
- * of the child window expression, transpose them.
+ * Reorder a stack of `Window` operators so that the fewest possible exchanges
are inserted
+ * for it. Window operators only append their output columns and each of them
+ * (re-)partitions/(re-)sorts its input independently of its position in the
stack, so two
+ * windows can be swapped freely as long as neither references the other's
output. Physically,
+ * a window rides the exchange created for the nearest window below it iff
that exchange's key
+ * set is a subset of the window's partition spec (`HashPartitioning`
satisfying
Review Comment:
Spec length is not a valid subset ordering. Duplicate keys break the claimed
exchange minimum. [a,b] → [a,a] remains unchanged because both specs have
length two, requiring two exchanges. Reversing them requires one because
HashPartitioning(a,a) satisfies relaxed ClusteredDistribution(a,b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
