wangyum opened a new pull request, #57248: URL: https://github.com/apache/spark/pull/57248
### What changes were proposed in this pull request? This PR fixes `BatchScanExec.doCanonicalize` to use `QueryPlan.normalizeExpressions` instead of `QueryPlan.normalizePredicates` when canonicalizing `keyGroupedPartitioning` expressions. `normalizePredicates` combines expressions with `And`, canonicalizes, then splits back — which can reorder the expressions. `normalizeExpressions` canonicalizes each expression individually, preserving the original order. ### Why are the changes needed? `BatchScanExec.doCanonicalize` previously used `QueryPlan.normalizePredicates` on `keyGroupedPartitioning`, which combines the partition expressions with `And`, canonicalizes, then splits back. This reordering causes a mismatch between expression data types and partition key row values, leading to a `ClassCastException` at runtime. For example, if `keyGroupedPartitioning` is `[id (IntegerType), data (StringType)]`, canonicalization could return `[data (StringType), id (IntegerType)]`, and partition values no longer align with their corresponding expressions. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added a regression test in `KeyGroupedPartitioningSuite`: `SPARK-58120: doCanonicalize preserves keyGroupedPartitioning expression order`, which verifies that the canonicalized `keyGroupedPartitioning` expressions preserve the same order and data types as the original. ### Was this patch authored or co-authored using generative AI tooling? Authored with assistance by GLM 5.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
