Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/20276#discussion_r162259637
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -730,12 +733,28 @@ class Analyzer(
right
case Some((oldRelation, newRelation)) =>
val attributeRewrites =
AttributeMap(oldRelation.output.zip(newRelation.output))
+ // If we de-duplicated an `AnalysisBarrier`, then we should only
replace
+ // `AttributeReference` that refers to this `AnalysisBarrier`.
+ val barrierId = oldRelation match {
+ case b: AnalysisBarrier => Some(b.id)
+ case _ => None
+ }
right transformUp {
case r if r == oldRelation => newRelation
} transformUp {
case other => other transformExpressions {
- case a: Attribute =>
- dedupAttr(a, attributeRewrites)
+ case a: AttributeReference =>
+ // Only replace `AttributeReference` when the
de-duplicated relation is not a
+ // `AnalysisBarrier`, or this `AttributeReference` is not
associated with any
+ // `AnalysisBarrier`, or this `AttributeReference` refers
to the de-duplicated
+ // `AnalysisBarrier`, i.e. barrierId matches.
+ if (barrierId.isEmpty ||
!a.metadata.contains(AnalysisBarrier.metadataKey) ||
+ barrierId.get ==
a.metadata.getLong(AnalysisBarrier.metadataKey)) {
+ dedupAttr(a, attributeRewrites)
--- End diff --
Looks like it is the same as:
```scala
// When we de-duplicated an `AnalysisBarrier` and this `AttributeReference`
is associated with other
// `AnalysisBarrier` different than the de-duplicated one, we don't replace
it.
val notToReplace = barrierId.map { id =>
a.metadata.contains(AnalysisBarrier.metadataKey) &&
id != a.metadata.getLong(AnalysisBarrier.metadataKey)
}.getOrElse(false)
if (notToReplace) {
a
} else {
dedupAttr(a, attributeRewrites)
}
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
