jianyexi opened a new pull request, #12948:
URL: https://github.com/apache/gluten/pull/12948
## What changes are proposed in this pull request?
Enable the parent-managed `flatten-maven-plugin` for `backends-velox` so its
installed and published POM records the Arrow version and scopes selected by
the active Spark profile.
Without flattening, the installed backend POM retains `${arrow.version}`. A
standalone Spark 4.0 or 4.1 module build can then resolve that property from
the parent default (Arrow 15.0.0) while Spark supplies other Arrow components
at 18.x. This produced a mixed classpath and `NoClassDefFoundError:
ArrowType$Utf8View` in Spark 4.1 tests. Full reactor builds mask the defect
because the active Spark profile remains available.
The new regression checker validates:
- Spark 3.3, 3.4, and 3.5 remain on Arrow 15.0.0 with the existing
runtime/compile scopes.
- Spark 4.0 resolves Arrow 18.1.0 with provided scope.
- Spark 4.1 resolves Arrow 18.3.0 with provided scope.
- Flattened metadata contains literal values rather than unresolved Arrow
properties.
- Maven's real flatten-to-install lifecycle installs the same flattened
Spark 4.1 POM.
- The checker preserves the source POM, existing build outputs, and the
user's Maven repository.
This changes published Maven metadata only; it does not change Gluten
runtime behavior.
## How was this patch tested?
- `dev/check-backends-velox-flattened-pom.sh`
- All five Spark profiles passed.
- Spark 4.1 lifecycle-generated and installed POMs were byte-identical.
- Final run completed in 116 seconds.
- Spark 3.5 reactor dependency tree:
- `arrow-memory-unsafe:15.0.0:runtime`
- `arrow-memory-core:15.0.0:compile`
- `arrow-vector:15.0.0:compile`
- Spark 4.1 reactor dependency tree:
- `arrow-memory-unsafe:18.3.0:provided`
- `arrow-memory-core:18.3.0:provided`
- `arrow-vector:18.3.0:provided`
- `./.github/workflows/util/check.py header branch`
- `bash -n dev/check-backends-velox-flattened-pom.sh`
- XML parsing of `backends-velox/pom.xml`
- `git diff --cached --check`
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI 1.0.80
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
