ulysses-you opened a new pull request, #58273: URL: https://github.com/apache/spark/pull/58273
### What changes were proposed in this pull request? `KeyedPartitioning.createShuffleSpec` now sorts the distinct projected partition keys (via `toGrouped`) so they follow the same natural ascending ordering as `GroupPartitionsExec`. ### Why are the changes needed? SPARK-56877 added a check in `PartitioningCollection.fromPartitionings` requiring all `KeyedPartitioning`s to share equal `partitionKeys`. In a storage-partitioned join whose join keys are a subset of the partition keys (e.g. a v2 table partitioned by `[dt, bucket(16, c1)]` joined on `c1`), with `spark.sql.sources.v2.bucketing.shuffle.enabled` enabled so only the non-keyed side is re-shuffled, the keyed side's projected keys are sorted by `GroupPartitionsExec` while `createShuffleSpec` kept them in first-occurrence order. The two sides then carry the same keys in different orders and the query fails with: ``` java.lang.IllegalArgumentException: requirement failed: All KeyedPartitionings in a PartitioningCollection must have equal partitionKeys ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added a regression test in `KeyGroupedPartitioningSuite` (`SPARK-56877: v2 bucketed table with subset join keys joining v1 table`) that reproduces the failure and passes with the fix. Also ran `KeyGroupedPartitioningSuite`, `EnsureRequirementsSuite`, `GroupPartitionsExecSuite`, and `ProjectedOrderingAndPartitioningSuite`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
