ulysses-you commented on PR #58884: URL: https://github.com/apache/spark/pull/58884#issuecomment-5773066696
Rebased onto master, which now carries SPARK-59396: `partitionKeyOrdering.enabled` and `preserveKeyOrderingOnCoalesce.enabled` default to true, so the ordering a scan derives from its partition keys is the grouping these tests are about, and a regrouping reports the key ordering it keeps. The tests now assert the default shapes: the pair folds with no sort at all, the regrouping reading the scan, and the narrowed case keeps the partial aggregate's sort under the regrouping. Two shapes need a sort between the pair, which needs the regrouping not to report its ordering, so the bail test and the bail arm set `preserveKeyOrderingOnCoalesce.enabled=false` for it, and the arm that covers the sort going with the partial aggregate sets both orderings off, the configuration that shape needs. `ReplaceHashWithSortAgg` plans the combined aggregate sort-based once the regrouping reports an ordering that satisfies the grouping, so the `collect_set` arm asserts the complete mode and the answer rather than the aggregate kind. Ran `KeyGroupedPartitioningSuite` and `GroupPartitionsExecSuite` (225) and the nine related suites (393), 618 in total, no failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
