[
https://issues.apache.org/jira/browse/SPARK-39259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547692#comment-17547692
]
Josh Rosen commented on SPARK-39259:
------------------------------------
This looks like it might be a fix for a correctness issue? If so, we should
probably backport this change to maintenance branches for the other
currently-supported Spark versions.
> Timestamps returned by now() and equivalent functions are not consistent in
> subqueries
> --------------------------------------------------------------------------------------
>
> Key: SPARK-39259
> URL: https://issues.apache.org/jira/browse/SPARK-39259
> Project: Spark
> Issue Type: Bug
> Components: Optimizer
> Affects Versions: 3.2.1
> Environment: Reproduced in the Spark Shell on the current 3.4.0
> snapshot
> Reporter: Jan-Ole Sasse
> Assignee: Jan-Ole Sasse
> Priority: Major
> Fix For: 3.3.0, 3.4.0
>
>
> Timestamp evaluation in not consistent across subqueries. As an example for
> the Spark Shell
>
> {code:java}
> sql("SELECT * FROM (SELECT 1) WHERE now() IN (SELECT now())").collect() {code}
> Returns an empty result.
>
> The root cause is that
> [ComputeCurrentTime|https://github.com/apache/spark/blob/c91c2e9afec0d5d5bbbd2e155057fe409c5bb928/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala#L74]
> does not iterate into subqueries
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
