Weichen Xu created SPARK-28621:
----------------------------------
Summary: CheckCartesianProducts do not work correctly
Key: SPARK-28621
URL: https://issues.apache.org/jira/browse/SPARK-28621
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.3
Reporter: Weichen Xu
CheckCartesianProducts do not work correctly.
There're some cases:
providing:
{code}
spark.range(2).createOrReplaceTempView("sm1") // can be broadcast
spark.range(50000000).createOrReplaceTempView("bg1") // cannot be broadcast
spark.range(60000000).createOrReplaceTempView("bg2") // cannot be broadcast
{code}
and suppose `spark.sql.crossJoin.enabled=false` (by default)
1) Some join could be convert to broadcast nested loop join, but
CheckCartesianProducts raise error. e.g.
{code}
select sm1.id, bg1.id from bg1 join sm1 where sm1.id < bg1.id
{code}
2) Some join will run by CartesianJoin but CheckCartesianProducts DO NOT raise
error. e.g.
{code}
select sm1.id, bg1.id from bg1 join sm1 where sm1.id < bg1.id
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
