[ 
https://issues.apache.org/jira/browse/SPARK-61467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Max Gekk updated SPARK-61467:
-----------------------------
    Description: 
When a rule replaces a node, TreeNode.transformDown/transformUp call 
copyTagsFrom on the expression the rule returns, and 
SupportQueryContext.copyTagsFrom sets its mutable queryContext field in place. 
That instance can be shared with other plans: a temp view's stored analyzed 
plan, or a parent Dataset, since QueryExecution clones the plan nodes but not 
their expressions. So the error of one query can report the context of another. 
With ANSI on:

{code:scala}
sql("SELECT CAST(CAST(id - 2147483648 AS INT) AS INTERVAL MONTH) / -1 AS d FROM 
range(1)")
  .createOrReplaceTempView("v")
sql("SELECT CAST(d AS INTERVAL YEAR TO MONTH) FROM v").collect()  // (1)
sql("SELECT d FROM v").collect()                                   // (2)
{code}

In (1), SimplifyCasts returns the view's DivideYMInterval instance in place of 
the same-type cast, which copies the cast's context into it. (2) then fails 
with the fragment CAST(d AS INTERVAL YEAR TO MONTH) and the SQL text of (1), 
with whole-stage codegen on and off. The same happens with a Dataset: 
base.select($"d".cast("interval year to month")) changes the context that a 
later base.collect() reports.

This affects every SupportQueryContext expression, such as ANSI Add; the 
interval division joined them in SPARK-61352. copyTagsFrom should not change an 
expression that other plans can share, e.g. by returning a copy with the new 
context.

> SupportQueryContext.copyTagsFrom overwrites the query context of an 
> expression shared with other plans
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61467
>                 URL: https://issues.apache.org/jira/browse/SPARK-61467
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 5.0.0
>            Reporter: Max Gekk
>            Priority: Major
>
> When a rule replaces a node, TreeNode.transformDown/transformUp call 
> copyTagsFrom on the expression the rule returns, and 
> SupportQueryContext.copyTagsFrom sets its mutable queryContext field in 
> place. That instance can be shared with other plans: a temp view's stored 
> analyzed plan, or a parent Dataset, since QueryExecution clones the plan 
> nodes but not their expressions. So the error of one query can report the 
> context of another. With ANSI on:
> {code:scala}
> sql("SELECT CAST(CAST(id - 2147483648 AS INT) AS INTERVAL MONTH) / -1 AS d 
> FROM range(1)")
>   .createOrReplaceTempView("v")
> sql("SELECT CAST(d AS INTERVAL YEAR TO MONTH) FROM v").collect()  // (1)
> sql("SELECT d FROM v").collect()                                   // (2)
> {code}
> In (1), SimplifyCasts returns the view's DivideYMInterval instance in place 
> of the same-type cast, which copies the cast's context into it. (2) then 
> fails with the fragment CAST(d AS INTERVAL YEAR TO MONTH) and the SQL text of 
> (1), with whole-stage codegen on and off. The same happens with a Dataset: 
> base.select($"d".cast("interval year to month")) changes the context that a 
> later base.collect() reports.
> This affects every SupportQueryContext expression, such as ANSI Add; the 
> interval division joined them in SPARK-61352. copyTagsFrom should not change 
> an expression that other plans can share, e.g. by returning a copy with the 
> new context.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to