rdtr opened a new pull request, #13265: URL: https://github.com/apache/gluten/pull/13265
## What changes are proposed in this pull request? This replaces #12165, which @philo-he approved but which was closed as stale before it was merged. It is rebased on the current main with no other changes. Enables `subquery/exists-subquery/exists-orderby-limit.sql` in the Spark 4.1 SQL query tests. It is commented out with `// TODO: fix on Spark-4.1` because it fails during physical planning: ``` java.lang.AssertionError: assertion failed: No plan for LocalLimit (1 + 2) ``` `GlutenSQLQueryTestSuite` excludes `ConvertToLocalRelation`, `ConstantFolding` and `NullPropagation` to force queries through Gluten's offload paths. For `EXISTS (... LIMIT 1 OFFSET 2)`, Spark's `LimitPushDown` rewrites the plan to `LocalLimit(Add(1, 2), ...)` and relies on `ConstantFolding` to fold the `Add`. `BasicOperators` only plans `LocalLimit` with an integer literal, so planning fails when `ConstantFolding` is excluded. This PR adds a `--SET` directive to this file that excludes only `ConvertToLocalRelation`, so `ConstantFolding` runs for this file. `NullPropagation` is re-enabled too, because the test framework's `--SET` parser splits values on commas and can't take a list of rules. The Spark fix is SPARK-57125 (https://github.com/apache/spark/pull/59194): `LimitPushDown` will produce the folded literal itself. Once Gluten uses a Spark version with that fix, the `--SET` directive can be removed. Related issue: #11916 ## How was this patch tested? As in #12165: without the `--SET` directive the test fails with the error above, and with it the test passes. The Spark 4.1 UT CI runs this file. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
