Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/14132#discussion_r72010521
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1774,6 +1775,49 @@ class Analyzer(
}
/**
+ * Substitute Hints.
+ * - BROADCAST/BROADCASTJOIN/MAPJOIN match the closest table with the
given name parameters.
+ *
+ * This rule substitutes `UnresolvedRelation`s in `Substitute` batch
before `ResolveRelations`
+ * rule is applied. Here are two reasons.
+ * - To support `MetastoreRelation` in Hive module.
+ * - To reduce the effect of `Hint` on the other rules.
+ *
+ * After this rule, it is guaranteed that there exists no unknown `Hint`
in the plan.
+ * All new `Hint`s should be transformed into concrete Hint classes
`BroadcastHint` here.
+ */
+ object SubstituteHints extends Rule[LogicalPlan] {
--- End diff --
Hi, @cloud-fan . Thank you again.
Yes. That's true. So, at my first commit of this PR, I actually implemented
to attach just the concrete `BroadcastHint` to the relations in
`AstBuilder.scala`. But, there were some advices to
- Move out the logic from the parser module (for preventing future updates
on parser module)
- Generalize the hint syntax like the current code.
Since a general `Hint` is attached on a logical plan not a table relation,
we can substitute any other plans (filter/project/aggregator), too. The
following is my previous comment about this situation. The child of hint is
`JOIN`.
> Yep. I grasp your point. First, the given relation is just a logical
plan, not a table. In the following case, the relation is Join.
> SELECT /*+ MAPJOIN(parquet_t0) */ * FROM parquet_t0, parquet_t1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
