Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20276#discussion_r162259637 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -730,12 +733,28 @@ class Analyzer( right case Some((oldRelation, newRelation)) => val attributeRewrites = AttributeMap(oldRelation.output.zip(newRelation.output)) + // If we de-duplicated an `AnalysisBarrier`, then we should only replace + // `AttributeReference` that refers to this `AnalysisBarrier`. + val barrierId = oldRelation match { + case b: AnalysisBarrier => Some(b.id) + case _ => None + } right transformUp { case r if r == oldRelation => newRelation } transformUp { case other => other transformExpressions { - case a: Attribute => - dedupAttr(a, attributeRewrites) + case a: AttributeReference => + // Only replace `AttributeReference` when the de-duplicated relation is not a + // `AnalysisBarrier`, or this `AttributeReference` is not associated with any + // `AnalysisBarrier`, or this `AttributeReference` refers to the de-duplicated + // `AnalysisBarrier`, i.e. barrierId matches. + if (barrierId.isEmpty || !a.metadata.contains(AnalysisBarrier.metadataKey) || + barrierId.get == a.metadata.getLong(AnalysisBarrier.metadataKey)) { + dedupAttr(a, attributeRewrites) --- End diff -- Looks like it is the same as: ```scala // When we de-duplicated an `AnalysisBarrier` and this `AttributeReference` is associated with other // `AnalysisBarrier` different than the de-duplicated one, we don't replace it. val notToReplace = barrierId.map { id => a.metadata.contains(AnalysisBarrier.metadataKey) && id != a.metadata.getLong(AnalysisBarrier.metadataKey) }.getOrElse(false) if (notToReplace) { a } else { dedupAttr(a, attributeRewrites) } ```
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org