Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12040#discussion_r58179063
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/Window.scala ---
@@ -885,11 +886,21 @@ private[execution] object AggregateProcessor {
val evaluateExpressions =
mutable.Buffer.fill[Expression](ordinal)(NoOp)
val imperatives = mutable.Buffer.empty[ImperativeAggregate]
+ // SPARK-14244: `SizeBasedWindowFunction`s are firstly created on
driver side and then
+ // serialized to executor side. These functions all reference a global
singleton window
+ // partition size attribute reference, i.e.,
`SizeBasedWindowFunction.n`. Here we must collect
+ // the singleton instance created on driver side instead of using
executor side
+ // `SizeBasedWindowFunction.n` to avoid binding failure caused by
mismatching expression ID.
+ val partitionSize = {
--- End diff --
Yeah, saw discussion between you and Yin above. This improves readability.
TBH, I was also quite confused while updating this part of code. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
