Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/20345#discussion_r175971417
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -84,19 +84,49 @@ object ReorderJoin extends Rule[LogicalPlan] with
PredicateHelper {
}
}
+ // Extract a list of logical plans to be joined for join-order
comparisons.
+ // Since `ExtractFiltersAndInnerJoins` handles left-deep trees only,
this function have
+ // the same strategy to extract the plan list.
+ private def extractLeftDeepInnerJoins(plan: LogicalPlan):
Seq[LogicalPlan] = plan match {
+ case j @ Join(left, right, _: InnerLike, _) => right +:
extractLeftDeepInnerJoins(left)
+ case p @ Project(_, j @ Join(_, _, _: InnerLike, _)) =>
extractLeftDeepInnerJoins(j)
+ case _ => Seq(plan)
+ }
+
+ private def checkSameJoinOrder(plan1: LogicalPlan, plan2: LogicalPlan):
Boolean = {
+ extractLeftDeepInnerJoins(plan1) == extractLeftDeepInnerJoins(plan2)
+ }
+
+ private def mayCreateOrderedJoin(
+ originalPlan: LogicalPlan,
+ input: Seq[(LogicalPlan, InnerLike)],
+ conditions: Seq[Expression]): LogicalPlan = {
+ val orderedJoins = createOrderedJoin(input, conditions)
+ if (!checkSameJoinOrder(orderedJoins, originalPlan)) {
--- End diff --
If we don't have this check, `operatorOptimizationRuleSet` reaches
`fixedPoint` because `ReorderJoin` is re-applied in the same join trees every
time the optimization rule batch invoked. This case does not happen in the
master because reordered joins have `Project` in internal nodes (`Project`
added by following optimization rules, e.g., `ColumnPruning`) and this plan
structure guards this case.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
