huanliwang-db commented on code in PR #58592:
URL: https://github.com/apache/spark/pull/58592#discussion_r3961824958
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveDeduplicate.scala:
##########
@@ -89,6 +89,24 @@ object ResolveDeduplicate extends Rule[LogicalPlan] {
}
}
+ /**
+ * Recomputes batch and streaming deduplication keys while preserving which
metadata columns were
+ * visible at the original deduplication boundary. Analyzer rules may add
metadata columns to the
+ * child later to satisfy downstream references, but those columns must not
silently become keys.
+ */
+ def recomputeKeysPreservingMetadataBoundary(
Review Comment:
is it possible because of introducing the `_metadata`, the ordering of the
original keys are changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
