[ https://issues.apache.org/jira/browse/SPARK-29145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takeshi Yamamuro resolved SPARK-29145. -------------------------------------- Fix Version/s: 3.0.0 Assignee: angerszhu Resolution: Fixed Resolved by [https://github.com/apache/spark/pull/25854|https://github.com/apache/spark/pull/25854#] > Spark SQL cannot handle "NOT IN" condition when using "JOIN" > ------------------------------------------------------------ > > Key: SPARK-29145 > URL: https://issues.apache.org/jira/browse/SPARK-29145 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.1.3, 2.2.3, 2.3.4, 2.4.4 > Reporter: Dezhi Cai > Assignee: angerszhu > Priority: Minor > Fix For: 3.0.0 > > > sample sql: > {code} > spark.range(10).createOrReplaceTempView("A") > spark.range(10).createOrReplaceTempView("B") > spark.range(10).createOrReplaceTempView("C") > sql("""select * from A inner join B on A.id=B.id and A.id NOT IN (select id > from C)""") > {code} > > {code} > org.apache.spark.sql.AnalysisException: Table or view not found: C; line 1 > pos 74; > 'Project [*] > +- 'Join Inner, ((id#0L = id#2L) AND NOT id#0L IN (list#6 [])) > : +- 'Project ['id] > : +- 'UnresolvedRelation [C] > :- SubqueryAlias `a` > : +- Range (0, 10, step=1, splits=Some(12)) > +- SubqueryAlias `b` > +- Range (0, 10, step=1, splits=Some(12)) > at > org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$1(CheckAnalysis.scala:94) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$1$adapted(CheckAnalysis.scala:89) > at > org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:155) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1(TreeNode.scala:154) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1$adapted(TreeNode.scala:154) > at scala.collection.immutable.List.foreach(List.scala:392) > at > org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:154) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis.checkAnalysis(CheckAnalysis.scala:89) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis.checkAnalysis$(CheckAnalysis.scala:86) > at > org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:120) > ... > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org