marin-ma opened a new issue, #11828: URL: https://github.com/apache/gluten/issues/11828
### Backend VL (Velox) ### Bug description Currently, Gluten uses the configuration `spark.gluten.sql.columnar.cudf=true` to enable GPU support. When running on a hybrid cluster, CPU nodes will fail if this configuration is set. We should disable the GPU code by checking whether the node has the CUDA runtime and available devices. Another issue that needs to be resolved is that we currently pass the cudf configurations `spark.gluten.sql.columnar.cudf` and `spark.gluten.sql.columnar.backend.velox.cudf.enableTableScan` to the session configuration at runtime for the WholeStageResultIterator. However, these are immutable configurations. We should remove them from the runtime session configuration and instead set their values from the backend configuration. ### Gluten version _No response_ ### Spark version None ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs ```bash ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
