[
https://issues.apache.org/jira/browse/SPARK-40615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-40615:
---------------------------------
Description:
Currently, we don't check if the join condition after subquery decorrelation is
resolved or not. This can lead to confusing error messages to users. We should
throw proper exceptions when one or more join conditions after decorrelation
are not resolved.
For example:
{code:java}
create temp view v1(x) as values from_json('{"a":1, "b":2}', 'map<string,int>')
t(x)
select (select a + a from (select upper(x['a']) as a)) from v1
After applying rule
org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates in batch
Pullup Correlated Expressions, the structural integrity of the plan is
broken.{code}
was:
Currently, we don't check if the join condition after subquery decorrelation is
resolved or not. This can lead to confusing error messages to users. We should
throw proper exceptions when one or more join conditions after decorrelation
are not resolved.
For example:
{code:java}
create temp view v1(x) as values from_json('{"a":1, "b":2}', 'map<string,int>')
t(x)
select (select a + a from (select upper(x['a']) as a)) from v1
After applying rule
org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates in batch
Pullup Correlated Expressions, the structural integrity of the plan is
broken.{code}
> Check unsupported data type when decorrelating subqueries
> ---------------------------------------------------------
>
> Key: SPARK-40615
> URL: https://issues.apache.org/jira/browse/SPARK-40615
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: Allison Wang
> Priority: Major
>
> Currently, we don't check if the join condition after subquery decorrelation
> is resolved or not. This can lead to confusing error messages to users. We
> should throw proper exceptions when one or more join conditions after
> decorrelation are not resolved.
> For example:
> {code:java}
> create temp view v1(x) as values from_json('{"a":1, "b":2}',
> 'map<string,int>') t(x)
> select (select a + a from (select upper(x['a']) as a)) from v1
> After applying rule
> org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates in batch
> Pullup Correlated Expressions, the structural integrity of the plan is
> broken.{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
