Jungtaek Lim created SPARK-36157:
------------------------------------
Summary: TimeWindow expression: apply filter before project
Key: SPARK-36157
URL: https://issues.apache.org/jira/browse/SPARK-36157
Project: Spark
Issue Type: Improvement
Components: SQL, Structured Streaming
Affects Versions: 3.3.0
Reporter: Jungtaek Lim
https://github.com/apache/spark/blob/4dfd266b27fea6954593c6b9e3a2819b290f0aec/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L3901-L3906
In the case of tumbling window, we apply project and then filter, while filter
is not dependent to project. We can just swap two operator to ensure less rows
would be projected if there're some rows being filtered out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
