akshaytayal opened a new pull request, #13435: URL: https://github.com/apache/gluten/pull/13435
## What changes are proposed in this pull request? `BasicScanExecTransformer` and `FilterExecTransformerBase` recombine the split filter conjuncts with `reduceLeftOption(And)`. That builds a left-deep tree whose depth equals the number of conjuncts, and the recursive `ExpressionConverter` throws `StackOverflowError` for large filters. Spark 4.2's new test `SPARK-56134: Codegen working for empty output` (1000 conjuncts) aborted the whole Spark 4.2 UT group, so #13163 disabled `GlutenWholeStageCodegenSuite` with `@Ignore`. - Add `FilterHandler.combineConjuncts`, which uses Spark's `PredicateHelper.buildBalancedPredicate`. It keeps the conjunct order, so results are unchanged, and the depth becomes log(N). Use it in the scan and filter transformers. - Remove the `@Ignore` from the Spark 4.2 `GlutenWholeStageCodegenSuite`. - The new Spark 4.2 FilterExec CSE tests (SPARK-56431, 3x SPARK-56032) assert on vanilla `WholeStageCodegenExec` and its generated Java, which Gluten replaces. Exclude them and add Gluten variants that check the filter is offloaded to a `WholeStageTransformer`, keeping the result checks with CSE on and off. Part of #13179 (umbrella #13181). ## How was this patch tested? Locally on Spark 4.2: - `GlutenWholeStageCodegenSuite`: 46 succeeded, 0 failed. Before the fix, the run aborted with `StackOverflowError` at SPARK-56134. - `GlutenParquetV1FilterSuite` / `GlutenParquetV2FilterSuite`: 84 succeeded, 0 failed. - backends-velox `VeloxScanSuite`, `VeloxTPCHV1Suite`, `VeloxTPCHV2Suite`: 54 succeeded, 0 failed. CI covers the other Spark versions and backends. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: GitHub Copilot CLI (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
