[ 
https://issues.apache.org/jira/browse/SPARK-42588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17693766#comment-17693766
 ] 

Apache Spark commented on SPARK-42588:
--------------------------------------

User 'zml1206' has created a pull request for this issue:
https://github.com/apache/spark/pull/40182

> collapse two adjacent windows with the equivalent partition/order expression
> ----------------------------------------------------------------------------
>
>                 Key: SPARK-42588
>                 URL: https://issues.apache.org/jira/browse/SPARK-42588
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.2.3, 3.3.2
>            Reporter: zhuml
>            Priority: Major
>
> Extend the CollapseWindow rule to collapse Window nodes with the equivalent 
> partition/order expressions
> {code:java}
> Seq((1, 1), (2, 2)).toDF("a", "b")
>       .withColumn("max_b", expr("max(b) OVER (PARTITION BY abs(a))"))
>       .withColumn("min_b", expr("min(b) OVER (PARTITION BY abs(a))"))
> == Optimized Logical Plan ==
> before
> Project [a#7, b#8, max_b#11, min_b#17]
> +- Window [min(b#8) windowspecdefinition(_w0#19, 
> specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) 
> AS min_b#17], [_w0#19]
>    +- Project [a#7, b#8, max_b#11, abs(a#7) AS _w0#19]
>       +- Window [max(b#8) windowspecdefinition(_w0#13, 
> specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) 
> AS max_b#11], [_w0#13]
>          +- Project [_1#2 AS a#7, _2#3 AS b#8, abs(_1#2) AS _w0#13]
>             +- LocalRelation [_1#2, _2#3]
> after
> Project [a#7, b#8, max_b#11, min_b#17]
> +- Window [max(b#8) windowspecdefinition(_w0#13, 
> specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) 
> AS max_b#11, min(b#8) windowspecdefinition(_w0#13, 
> specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) 
> AS min_b#17], [_w0#13]
>    +- Project [_1#2 AS a#7, _2#3 AS b#8, abs(_1#2) AS _w0#13]
>       +- LocalRelation [_1#2, _2#3]
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to