QCLyu commented on PR #11351:
URL:
https://github.com/apache/incubator-gluten/pull/11351#issuecomment-3733421723
## Summary of most recent changes:
### Main deprecation commit (10bed0dd7 - "deprecate Spark-3.2")
#### Removed Spark 3.2 support:
- Deleted shims/spark32/ directory (entire Spark 3.2 shim module)
- Removed spark-3.2 from Maven profiles in pom.xml and gluten-ut/pom.xml
- Removed Spark 3.2 CI test jobs from
.github/workflows/velox_backend_x86.yml:
- tpc-test-ubuntu-oom (Spark 3.2 specific)
- tpc-test-ubuntu-randomkill (Spark 3.2 specific)
- spark-test-spark32 (Spark 3.2 UT tests)
- spark-test-spark32-slow (Spark 3.2 slow tests)
- Removed spark-3.2 from build scripts:
- dev/buildbundle-veloxbe.sh
- dev/builddeps-veloxbe.sh
- dev/format-scala-code.sh
- dev/release/build-release.sh
- dev/release/package-release.sh
- dev/docker/Dockerfile.centos8-dynamic-build
- dev/docker/Dockerfile.centos9-dynamic-build
- Updated documentation to remove Spark 3.2 references
#### Updated (not deleted) — per reviewer feedback:
- tpc-test-centos8-uniffle job:
Changed from spark-3.2 → spark-3.3 in matrix
Updated commands to use spark-3.3
- tpc-test-ubuntu-2204-celeborn job:
Changed from spark-3.2 → spark-3.3 in matrix
Updated commands to use spark-3.3
- Test code updates:
backends-velox/src/test/scala/org/apache/gluten/execution/VeloxHashJoinSuite.scala:
* Removed isSparkVersionLE("3.2") check
* Fixed Scalastyle line length issue
### CI fixes commit (0861158ae - "Fixed workflow paths")
- Fixed 4 CI failures in .github/workflows/velox_backend_x86.yml:
Added SETUP environment variable (line 49):
SETUP: 'source .github/workflows/util/setup-helper.sh'
- Fixed tpc-test-centos8-uniffle Maven path (line 406):
Changed: $MVN_CMD → $GITHUB_WORKSPACE/$MVN_CMD
- Fixed tpc-test-ubuntu-2204-celeborn setup (lines 439-440):
Changed: $SETUP install_maven → source
.github/workflows/util/setup-helper.sh + install_maven
- Fixed tpc-test-ubuntu-2204-celeborn Maven path (line 467):
Changed: $MVN_CMD → $GITHUB_WORKSPACE/$MVN_CMD
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
