[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15652366#comment-15652366 ]
Xiangrui Meng commented on SPARK-18390: --------------------------------------- This is a bug because the user didn't ask a cartesian join. Anyway, this was fixed. > Optimized plan tried to use Cartesian join when it is not enabled > ----------------------------------------------------------------- > > Key: SPARK-18390 > URL: https://issues.apache.org/jira/browse/SPARK-18390 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.0.1 > Reporter: Xiangrui Meng > Assignee: Srinath > > {code} > val df2 = spark.range(1e9.toInt).withColumn("one", lit(1)) > val df3 = spark.range(1e9.toInt) > df3.join(df2, df3("id") === df2("one")).count() > {code} > throws > bq. org.apache.spark.sql.AnalysisException: Cartesian joins could be > prohibitively expensive and are disabled by default. To explicitly enable > them, please set spark.sql.crossJoin.enabled = true; > This is probably not the right behavior because it was not the user who > suggested using cartesian product. SQL picked it while knowing it is not > enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org