peter-toth commented on PR #58552: URL: https://github.com/apache/spark/pull/58552#issuecomment-5615001999
Rebased and all four comments answered, thank you both. **Rebased over #58531 (SPARK-59256) and #58591 (SPARK-59310)**, which both landed after the last push and both touch these files. Three conflicts: `otherSpec.numPartitions` beside this PR's key-space clause, #58531's re-nested reduce block against this PR's replacement of the SPARK-59176 exception, and #58591's two metric fields beside this PR's `dataTypes` and `layout` on `PartitionGrouping`. **Two of dongjoon-hyun's comments were defects in the diff, and both are fixed** in [`d839aee`](https://github.com/apache/spark/commit/d839aeeb3146f07d69b4cebda2b7ed9efdb2d9ff): - `project` built a fresh `KeyLayout` and so defaulted `mayContainUnknownPartitionKeys` to false, where `copy(...)` had carried it. Unreachable today, but the `createShuffleSpec` comment claimed otherwise. It carries it now. - `stringArgs` dropped the marker from the plan string, so the description's "No plan string changes" was wrong. The marker prints again, unconditionally, and the description is accurate as written. **ulysses-you's two non-blocking comments are both taken** in the same commit: the pruned-layout test pins its claim positively rather than on absence, and the type gate now records why an empty-versus-empty pair sharing a type is inert. 458 tests green: `DistributionSuite`, `ShuffleSpecSuite`, `KeyGroupedPartitioningSuite`, `GroupPartitionsExecSuite`, `EnsureRequirementsSuite`, `ProjectedOrderingAndPartitioningSuite`, `PlannerSuite`, `DataSourceV2CatalystRuntimeFilterSuite`. Lint clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
