[jira] [Assigned] (SPARK-42588) collapse two adjacent windows with the equivalent partition/order expression
[ https://issues.apache.org/jira/browse/SPARK-42588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42588: Assignee: Apache Spark > collapse two adjacent windows with the equivalent partition/order expression > > > Key: SPARK-42588 > URL: https://issues.apache.org/jira/browse/SPARK-42588 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.2.3, 3.3.2 >Reporter: zhuml >Assignee: Apache Spark >Priority: Major > > Extend the CollapseWindow rule to collapse Window nodes with the equivalent > partition/order expressions > {code:java} > Seq((1, 1), (2, 2)).toDF("a", "b") > .withColumn("max_b", expr("max(b) OVER (PARTITION BY abs(a))")) > .withColumn("min_b", expr("min(b) OVER (PARTITION BY abs(a))")) > == Optimized Logical Plan == > before > Project [a#7, b#8, max_b#11, min_b#17] > +- Window [min(b#8) windowspecdefinition(_w0#19, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS min_b#17], [_w0#19] >+- Project [a#7, b#8, max_b#11, abs(a#7) AS _w0#19] > +- Window [max(b#8) windowspecdefinition(_w0#13, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS max_b#11], [_w0#13] > +- Project [_1#2 AS a#7, _2#3 AS b#8, abs(_1#2) AS _w0#13] > +- LocalRelation [_1#2, _2#3] > after > Project [a#7, b#8, max_b#11, min_b#17] > +- Window [max(b#8) windowspecdefinition(_w0#13, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS max_b#11, min(b#8) windowspecdefinition(_w0#13, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS min_b#17], [_w0#13] >+- Project [_1#2 AS a#7, _2#3 AS b#8, abs(_1#2) AS _w0#13] > +- LocalRelation [_1#2, _2#3] > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42588) collapse two adjacent windows with the equivalent partition/order expression
[ https://issues.apache.org/jira/browse/SPARK-42588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42588: Assignee: (was: Apache Spark) > collapse two adjacent windows with the equivalent partition/order expression > > > Key: SPARK-42588 > URL: https://issues.apache.org/jira/browse/SPARK-42588 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.2.3, 3.3.2 >Reporter: zhuml >Priority: Major > > Extend the CollapseWindow rule to collapse Window nodes with the equivalent > partition/order expressions > {code:java} > Seq((1, 1), (2, 2)).toDF("a", "b") > .withColumn("max_b", expr("max(b) OVER (PARTITION BY abs(a))")) > .withColumn("min_b", expr("min(b) OVER (PARTITION BY abs(a))")) > == Optimized Logical Plan == > before > Project [a#7, b#8, max_b#11, min_b#17] > +- Window [min(b#8) windowspecdefinition(_w0#19, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS min_b#17], [_w0#19] >+- Project [a#7, b#8, max_b#11, abs(a#7) AS _w0#19] > +- Window [max(b#8) windowspecdefinition(_w0#13, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS max_b#11], [_w0#13] > +- Project [_1#2 AS a#7, _2#3 AS b#8, abs(_1#2) AS _w0#13] > +- LocalRelation [_1#2, _2#3] > after > Project [a#7, b#8, max_b#11, min_b#17] > +- Window [max(b#8) windowspecdefinition(_w0#13, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS max_b#11, min(b#8) windowspecdefinition(_w0#13, > specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) > AS min_b#17], [_w0#13] >+- Project [_1#2 AS a#7, _2#3 AS b#8, abs(_1#2) AS _w0#13] > +- LocalRelation [_1#2, _2#3] > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org