baibaichen opened a new issue, #13167:
URL: https://github.com/apache/gluten/issues/13167
## Problem
Native C++ unit-test executables link the production JNI shared libraries
instead of non-JNI implementation targets.
The current CMake dependency chain is:
```text
velox_plan_conversion_test -> libvelox.so -> libgluten.so
\-> libgluten.so (transitive PUBLIC link)
```
`libgluten.so` contains the common JNI wrapper and core implementation,
while `libvelox.so` contains the Velox JNI wrapper and backend implementation.
The vcpkg toolchain links `libstdc++` and `libgcc` statically into executables
and shared libraries. As a result, a native test process can contain
independent C++ runtime copies in the test executable, `libvelox.so`, and
`libgluten.so` while exchanging C++ objects across those DSO boundaries.
Symbol version scripts and `--exclude-libs` prevent unwanted symbol
interposition, but they do not prevent `std::shared_ptr`, exceptions, RTTI,
locale state, or object ownership from crossing the DSO boundary.
## Reproduction
With a Linux vcpkg static build:
```bash
./dev/package-vcpkg.sh --run_setup_script=OFF --spark_version=4.1
./cpp/build/velox/tests/velox_plan_conversion_test --gtest_list_tests
```
The test executable can terminate during discovery with:
```text
free(): invalid pointer
```
No test case needs to run; the failure occurs during process/DSO
initialization and GTest discovery.
The current build contains:
- 18 Velox/Delta test executables that depend on both `libvelox.so` and
`libgluten.so`
- 5 core test executables that depend on `libgluten.so`
- 5702 registered CTest cases across those executables
## Proposed direction
Separate implementation code from JNI facades:
```text
gluten_core implementation target
velox_backend implementation target
libgluten.so = gluten_core + common JNI wrapper
libvelox.so = velox_backend + Velox JNI wrapper + gluten_core dependency
native UT = test sources + implementation targets + GTest
```
The implementation targets may be object or static libraries so source files
are compiled once and native tests do not load the production JNI DSOs.
Keep separate JNI/Spark integration tests that exercise the actual
production chain:
```text
JVM -> libvelox.so -> libgluten.so
```
## Non-goals
- Do not globally remove `-static-libstdc++` or `-static-libgcc` from vcpkg
production artifacts.
- Do not treat switching all test-enabled package builds to dynamic C++
runtime as a fix.
- Do not include this refactor in unrelated Arrow/vcpkg dependency changes.
## Acceptance criteria
- Native core tests link the non-JNI core implementation target rather than
`libgluten.so`.
- Native Velox tests link non-JNI core/backend implementation targets rather
than `libvelox.so`/`libgluten.so`.
- JNI shared-library packaging and exported JNI symbols remain unchanged.
- Native CTest discovery and execution succeed with the vcpkg static-runtime
configuration.
- Existing JNI/Spark integration tests continue to validate production
shared-library loading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
