metanil commented on PR #55885:
URL: https://github.com/apache/spark/pull/55885#issuecomment-6375280694
@peter-toth thanks for the review. Pushed as `381a61a`
- **11**: done as suggested, `72fd1b4`.
- **12**: `2954e1a`, with one difference from your sketch. The pair-level
entry point is
`pairReducers` and returns a flat `(Option[Reducer], Option[Reducer])`,
`(None, None)` when either
direction threw, rather than an `Option` around the pair: both callers
treat "vetoed" and "neither
reducible" the same, so the outer `Option` only made each caller unwrap
it. Your fixture and test
are in: 48 of 48 rows, 16 with the veto removed.
- **13**: done as suggested, `6885e6e`. Also covers **20**.
- **14**: done as suggested, `ece7515`.
- **15**: agreed, deferring to a follow-up commit. The plan is
`Unimplemented` at DEBUG with "provides
no reducer onto ...", keeping the override hint only when every overload
the pair could use was
unimplemented, and `Threw` at WARN once per canonical name, then DEBUG.
- **16**: migration-guide entries in `b62321a`, plus a second entry for the
exact-type rule, since it
applies to the identity reduce and the one-side shuffle for every
transform, not only those with
literal parameters. I'll update the description the same way.
- **19**: `381a61a` makes `sameArgumentLayout` use identity's shapes, so
`bucket(8, s.a)` against
`bucket(4, s.b)` no longer reaches the connector. I haven't yet retired
the separate normalization
in `KeyedShuffleSpec.argumentShape`; will follow up.
- **17, 18, 21, 22, 23**: will follow up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
