rdtr opened a new pull request, #13187: URL: https://github.com/apache/gluten/pull/13187
## What changes are proposed in this pull request? This takes over #11908 by @baibaichen, which was approved but closed as stale after it got conflicts with main. It's rebased onto the current main with the same changes: - Enable `GlutenPythonDataSourceSuite`, `GlutenPythonUDFSuite` and `GlutenPythonUDTFSuite` on Spark 4.0 and 4.1, and `GlutenRowQueueSuite` on Spark 4.1. - `GlutenPythonUDFSuite`: replace `SPARK-48706` with a `testGluten` version, since the original test expects the Spark suite name in the call site. - `GlutenPythonUDTFSuite`: extend `PythonUDTFSuite` (it was an empty class). - `GlutenPythonDataSourceSuite` (Spark 4.1): add a `testGluten` version of the filter pushdown test. - `ArrowEvalPythonExecSuite`: run the 3 `arrow_udf` tests on Spark 4.1 too. - Install Python in the `spark40-slow` and `spark41-slow` CI jobs, using the same step as the other Spark 4.x jobs, so these tests run instead of being canceled. The only conflict was in `velox_backend_x86.yml`, where the slow jobs have since been split by test suite and download different artifacts. This PR adds only the Python install step to the current jobs. Related issue: #11550 ## How was this patch tested? On Spark 4.1 with Python 3.11, pyspark 3.5.5, pandas 2.2.3 and pyarrow 20.0.0 (the CI versions): - `GlutenPythonDataSourceSuite`, `GlutenPythonUDFSuite`, `GlutenPythonUDTFSuite`, `GlutenRowQueueSuite`: 50 passed, 0 canceled, 2 ignored (the excluded tests). - `ArrowEvalPythonExecSuite`: 4 passed. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
