Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21745#discussion_r202055806
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1165,15 +1173,19 @@ class Analyzer(
(newExprs, AnalysisBarrier(newChild))
case p: Project =>
+ // Resolving expressions against current plan.
val maybeResolvedExprs = exprs.map(resolveExpression(_, p))
+ // Recursively resolving expressions on the child of current
plan.
val (newExprs, newChild) =
resolveExprsAndAddMissingAttrs(maybeResolvedExprs, p.child)
- val missingAttrs = AttributeSet(newExprs) --
AttributeSet(maybeResolvedExprs)
+ // If some attributes used by expressions are resolvable only
on the rewritten child
+ // plan, we need to add them into original projection.
+ val missingAttrs = (AttributeSet(newExprs) --
p.outputSet).intersect(newChild.outputSet)
--- End diff --
what if we do not do the `.intersect(newChild.outputSet)`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
