This is an automated email from the ASF dual-hosted git repository.
SteNicholas pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 1f15bdcda4 [CELEBORN-2416] Export the C ABI from libceleborn_client.so
by dropping --exclude-libs,ALL
1f15bdcda4 is described below
commit 1f15bdcda40bcc9107d3f0b8ba64ad66c785cd0b
Author: Yu Gan <[email protected]>
AuthorDate: Tue Aug 11 20:19:33 2026 +0800
[CELEBORN-2416] Export the C ABI from libceleborn_client.so by dropping
--exclude-libs,ALL
### What changes were proposed in this pull request?
Drop `-Wl,--exclude-libs,ALL` from the `celeborn_client` link options on
ELF platforms, leaving the version script as the sole symbol-visibility
mechanism, and record at the call site why the flag must not come back.
### Why are the changes needed?
On ELF platforms the shared library is currently linked with both a version
script and `--exclude-libs,ALL`:
```cmake
target_link_options(celeborn_client PRIVATE
"-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/exports.map"
"-Wl,--exclude-libs,ALL")
```
`--exclude-libs,ALL` forces `STV_HIDDEN` on every symbol that comes out of
a static archive. The `celeborn_ffi_*` entry points live in
`libceleborn_ffi.a`, which is pulled into the shared library with
`--whole-archive` a few lines above, so they are exactly the symbols it hides.
Hidden visibility takes precedence over the version script, so the `global:`
entries in `exports.map` have no effect:
```
{
global:
extern "C++" { celeborn::*; };
celeborn_*;
celeborn_ffi_*;
local:
*;
};
```
The result is that the C ABI never reaches the dynamic symbol table of
`libceleborn_client.so`. A downstream binding that links the library by name
cannot resolve `celeborn_ffi_create_client` and friends — for the Rust crate
under `rust/`, which links it as `dylib=celeborn_client`, the link step fails
outright. The FFI shim is unreachable on Linux.
macOS is unaffected: it uses `-Wl,-exported_symbols_list` with
`exports.txt`, which has no equivalent flag.
Dropping the flag loses nothing, because the version script already does
what it was there for: the `local: *;` rule localizes every symbol that is not
`celeborn::*` or `celeborn_*`, so third-party symbols (folly, protobuf, glog,
abseil) stay hidden and cannot collide with the host process's own runtimes.
### Does this PR resolve a correctness bug?
- [ ] Yes
This is a build/packaging bug. It does not affect shuffle results; it makes
the C ABI unusable from a Linux build.
### Does this PR introduce _any_ user-facing change?
- [ ] Yes
No config, API or behaviour change. The exported symbol set of
`libceleborn_client.so` gains the `celeborn_ffi_*` entry points it was always
meant to expose; the set of hidden third-party symbols is unchanged.
### How was this patch tested?
Manual check on a Linux build — before the change
`celeborn_ffi_create_client` is absent from the dynamic symbol table, after it
is present and `T`:
```
$ nm -D --defined-only build/libceleborn_client.so | grep
celeborn_ffi_create_client
```
There is no CI step that inspects the exported symbols of the produced
library, so this is not covered by an automated check. If reviewers would like
one, I am happy to add an assertion to `.github/workflows/cpp_integration.yml`
in this PR or a follow-up.
The existing `Celeborn Cpp Integration Test` workflow still exercises the
build and the C++ unit tests.
Closes #3794 from yugan95/CELEBORN-2416.
Authored-by: Yu Gan <[email protected]>
Signed-off-by: Nicholas Jiang <[email protected]>
---
cpp/CMakeLists.txt | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
index 151a70f65d..ab194d5604 100644
--- a/cpp/CMakeLists.txt
+++ b/cpp/CMakeLists.txt
@@ -337,9 +337,16 @@ if(APPLE)
INSTALL_RPATH
"@loader_path;${HOMEBREW_PREFIX}/lib;${HOMEBREW_PREFIX}/opt/openssl@3/lib"
BUILD_WITH_INSTALL_RPATH TRUE)
else()
+ # The version script alone is enough: it puts everything except celeborn::*
+ # and celeborn_* into the `local:` bucket. Do NOT add -Wl,--exclude-libs,ALL
+ # on top of it -- that forces STV_HIDDEN on every symbol coming out of a
+ # static archive, and the celeborn_ffi_* entry points live in the
+ # libceleborn_ffi.a that is pulled in with --whole-archive above. Hidden
+ # visibility wins over the version script, so the C ABI would disappear from
+ # the .so's dynamic symbol table and downstream bindings (the Rust crate
+ # links this as `dylib=celeborn_client`) would fail to resolve it.
target_link_options(celeborn_client PRIVATE
- "-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/exports.map"
- "-Wl,--exclude-libs,ALL")
+ "-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/exports.map")
set_target_properties(celeborn_client PROPERTIES
INSTALL_RPATH "$ORIGIN"
BUILD_WITH_INSTALL_RPATH TRUE)
