andygrove opened a new pull request, #4960: URL: https://github.com/apache/datafusion-comet/pull/4960
## Which issue does this PR close? Part of #4142. ## Rationale for this change Spark 4.2.0 is now available in Maven Central, so the `spark-4.2` profile no longer has to target a preview. This PR moves the profile onto the released version and makes the main sources compile against it, without yet wiring up Spark's own SQL tests for 4.2. It is a smaller, compile-only slice of the larger bring-up in #4950, intended to land the version bump on its own. Spark 4.2.0 is not simply `4.2.0-preview4` with the qualifier dropped: it changes `SpecializedGetters` and repackages a dependency, so a couple of real reconciliations are needed to compile. ## What changes are included in this PR? **Build** - `spark.version` moves from `4.2.0-preview4` to `4.2.0`, and the WIP compile-only comment on the `spark-4.2` profile is dropped. - Spark 4.2.0 bundles its own copy of `org.apache.datasketches.memory.internal.ResourceImpl` inside `spark-catalyst`, colliding with the transitive `datasketches-memory` jar and failing the `BanDuplicateClasses` enforcer rule. That class is now ignored. - The stale Jetty comment that claimed the test-scope pin matched preview4's Jetty version is corrected. The pin itself is unchanged. **Shims** - Spark 4.2 removes `GeographyVal` / `GeometryVal` and replaces `getGeography` / `getGeometry` on `SpecializedGetters` with a single `getBinaryView` returning `BinaryView`. `CometInternalRowShim` can no longer be shared from the `spark-4.1+` source root, because the major, minor-plus and minor shim directories are all compile source roots (a `spark-4.2` copy would be a duplicate class, not an override). It is split into per-version `spark-4.1` and `spark-4.2` copies. This PR intentionally does not enable the Spark SQL tests, add CI jobs, or regenerate golden files for 4.2; that work is tracked separately in #4950. ## How are these changes tested? `./mvnw -Pspark-4.2 -DskipTests test-compile` builds the main and test sources against the released Spark 4.2.0. `./mvnw -Pspark-4.1 -DskipTests test-compile` still passes, confirming the shim split did not regress the 4.1 profile. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
