akshaytayal opened a new pull request, #13052:
URL: https://github.com/apache/gluten/pull/13052
### What
Adds the Apache Spark 4.2 build profile (`-Pspark-4.2`) and the
`shims/spark42`
layer, so Gluten can compile and run against Spark 4.2.0.
Part of the Spark 4.2 support effort (#12569). This is the shim layer; the
`gluten-ut/spark42` unit-test module follows in separate PRs.
### Details
- New `shims/spark42` module (`Spark42Shims` + `SparkShimProvider`)
implementing
the shim contract for Spark 4.2, including the SampleExec seed
(SPARK-53564) and
`KeyGroupedPartitioning` -> `KeyedPartitioning` (SPARK-53401) changes.
- `postDriverMetrics`, `SpecializedGetters.getBinaryView`, columnar-vector,
MemoryStream, PythonUDFRunner and BatchScan SPJ shims for Spark 4.2.
- Rebased on current `main`: the stale `PromotePrecision` shim and
`widerDecimalType` override are dropped (removed upstream in GLUTEN-12991
/ #13002).
### How was this patch tested
`shims/common` + `shims/spark42` compile under `-Pspark-4.2 -Pscala-2.13
-Pjava-17
-Pbackends-velox` (BUILD SUCCESS). Full Spark 4.2 UT validation lands with
the
gluten-ut/spark42 PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
