Allison Wang created SPARK-40615: ------------------------------------ Summary: Check unsupported data type when decorrelating subqueries Key: SPARK-40615 URL: https://issues.apache.org/jira/browse/SPARK-40615 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.4.0 Reporter: Allison Wang
Currently, we don't check if the join condition after subquery decorrelation is resolved or not. This can lead to confusing error messages to users. We should throw proper exceptions when one or more join conditions after decorrelation are not resolved. For example: {code:java} create temp view v1(x) as values from_json('{"a":1, "b":2}', 'map<string,int>') t(x) select (select a + a from (select upper(x['a']) as a)) from v1 After applying rule org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates in batch Pullup Correlated Expressions, the structural integrity of the plan is broken.{code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org