Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22326#discussion_r220436485 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1308,6 +1312,16 @@ object CheckCartesianProducts extends Rule[LogicalPlan] with PredicateHelper { if (SQLConf.get.crossJoinEnabled) { plan } else plan transform { + case j @ Join(_, _, _, condition) + if condition.isDefined && PullOutPythonUDFInJoinCondition.hasPythonUDF(condition.get) => + // if the crossJoinEnabled is false, a RuntimeException will be thrown later while + // the PythonUDF need to access both side of join, we throw firstly here for better + // readable information. + throw new AnalysisException(s"Detected the join condition:${j.condition} of this join " + --- End diff -- when will we hit it? If the python udf can't be pulled out because of join type problems, we already throw exception at `PullOutPythonUDFInJoinCondition`. Did I miss something?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org