ulysses-you commented on code in PR #58335:
URL: https://github.com/apache/spark/pull/58335#discussion_r3871609114
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala:
##########
@@ -1374,9 +1376,11 @@ case class KeyedShuffleSpec(
*
* @param other other key-grouped shuffle spec
*/
- def reducers(other: KeyedShuffleSpec): Option[Seq[Option[Reducer[_, _]]]] = {
+ def reducers(
+ other: KeyedShuffleSpec): Option[Seq[Option[(Reducer[_, _],
TransformExpression)]]] = {
val results =
partitioning.expressions.zip(other.partitioning.expressions).map {
- case (e1: TransformExpression, e2: TransformExpression) =>
e1.reducers(e2)
+ case (e1: TransformExpression, e2: TransformExpression) =>
+ e1.reducers(e2).map(reducer => (reducer, e1))
Review Comment:
And for shape 3 — please go ahead and take it as a follow-up as you offered;
we are happy to leave the both-sides-reduce shape (and the 12/8/6
stale-expression case) out of this PR. Happy to review your follow-up, and to
file a JIRA for it if that is useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
