szehon-ho opened a new pull request, #58905: URL: https://github.com/apache/spark/pull/58905
### What changes were proposed in this pull request? This follow-up expands the Spark 4.2.1 SQL migration guide entry for SPARK-59572. It explains that `PartitionPredicate` is sent during the second pass of iterative pushdown and adds concrete `SELECT` and metadata-only `DELETE` examples. This is the `branch-4.2` counterpart of https://github.com/apache/spark/pull/58902 and follows up https://github.com/apache/spark/pull/58851. ### Why are the changes needed? The existing entry states that evaluation failures now propagate, but does not make clear why returning `true` was a correctness bug. Once a data source accepts a static `PartitionPredicate`, Spark does not retain another evaluator for the condition. The same is true for metadata-only `DELETE`. The expanded examples show how the old behavior could return rows rejected by a filter or delete a partition that never matched the delete condition. The master documentation commit does not cleanly cherry-pick because `branch-4.2` does not contain the later-version migration sections. This PR applies only the relevant 4.2.1 change. ### Does this PR introduce _any_ user-facing change? No. This is a documentation-only clarification of the behavior fixed by SPARK-59572. ### How was this patch tested? Not run (documentation-only change). `git diff --check` passes. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
