ulysses-you opened a new pull request, #58460:
URL: https://github.com/apache/spark/pull/58460
### What changes were proposed in this pull request?
Follow-up to the `branch-4.3` backport of #58335 (commit 0d253fef083). Sets
`spark.sql.sources.v2.bucketing.requireAllClusterKeysForCoPartition` to false
in the `SPARK-59045: compatible transforms reduce data type with subset join
keys` test, which is what every other subset test in this suite already does on
this branch.
### Why are the changes needed?
The backport broke this test on `branch-4.3`:
```
- SPARK-59045: compatible transforms reduce data type with subset join keys
*** FAILED ***
List(Exchange hashpartitioning(id#L, 5), ENSURE_REQUIREMENTS, ...) was not
empty
storage-partitioned join should not shuffle
```
On `master` the test plans without the config because SPARK-58558 relaxed
`createKeyedShuffleSpec` from requiring the partition keys to exactly match the
join keys to requiring them to cover the join keys. SPARK-58558 is not on this
branch, so with the config at its default `createKeyedShuffleSpec` refuses the
subset shape, both sides are shuffled, and the no-shuffle assertion fails.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The failing test now passes; ran `KeyGroupedPartitioningSuite`.
### Was this patch authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
