This is an automated email from the ASF dual-hosted git repository.
yihua pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hudi-rs.git
The following commit(s) were added to refs/heads/main by this push:
new 8043ea15 build(ffi): forward the spill-rocksdb feature through
hudi-ffi (#849)
8043ea15 is described below
commit 8043ea15d2d1e61a2fbccd795feacda8b88001fb
Author: Y Ethan Guo <[email protected]>
AuthorDate: Wed Oct 28 11:45:54 2026 -0700
build(ffi): forward the spill-rocksdb feature through hudi-ffi (#849)
The C ABI crate took hudi with its defaults, so a consumer of the crate
got the merge map's on-disk tier (RocksDB, bindgen, libclang) with no way
to opt out, since a consumer can only drop a feature its direct dependency
exposes. Forward it the way hudi-datafusion and hudi do: default on,
hudi taken with default-features = false.
The no-spill-backend CI job now checks the crate without the feature and
includes it in the dependency-graph assertion.
---
.github/workflows/ci.yml | 16 +++++++++++-----
AGENTS.md | 15 ++++++++-------
crates/ffi/Cargo.toml | 8 +++++++-
3 files changed, 26 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2b860c07..1e3f5712 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -116,6 +116,10 @@ jobs:
run: cargo check -p hudi --no-default-features
- name: Check the umbrella crate without the spill backend, with
datafusion
run: cargo check -p hudi --no-default-features --features datafusion
+ # The C ABI crate reaches hudi-core through hudi and must forward the
+ # opt-out the same way.
+ - name: Check the C ABI crate without the spill backend
+ run: cargo check -p hudi-ffi --no-default-features
# The steps above prove the opt-out COMPILES. This proves it actually
drops
# the dependency, and unlike them it does not rest on libclang being
absent,
# so it keeps working if this job ever gains a C++ toolchain.
@@ -130,8 +134,8 @@ jobs:
# Under defaults the tier must be reachable. If this ever stops
# holding, the absence checks below are asserting nothing.
- for args in "" "--features datafusion"; do
- if ! out=$(cargo tree -p hudi $args -i rocksdb 2>&1) \
+ for args in "-p hudi" "-p hudi --features datafusion" "-p hudi-ffi";
do
+ if ! out=$(cargo tree $args -i rocksdb 2>&1) \
|| ! printf '%s\n' "$out" | grep -q '^rocksdb'; then
echo "cargo tree did not find rocksdb under default features:
'$args'"
echo "so the absence checks below cannot tell an opt-out from a
broken command"
@@ -143,9 +147,11 @@ jobs:
# With the opt-out it must be gone — and gone because the package is
# absent from the graph, which cargo says in those words, not because
# the command failed for some other reason.
- for args in "--no-default-features" "--no-default-features
--features datafusion"; do
- if out=$(cargo tree -p hudi $args -i rocksdb 2>&1); then
- echo "rocksdb is still reachable from hudi with: $args"
+ for args in "-p hudi --no-default-features" \
+ "-p hudi --no-default-features --features datafusion" \
+ "-p hudi-ffi --no-default-features"; do
+ if out=$(cargo tree $args -i rocksdb 2>&1); then
+ echo "rocksdb is still reachable with: $args"
printf '%s\n' "$out"
exit 1
elif ! printf '%s\n' "$out" | grep -q 'did not match any
packages'; then
diff --git a/AGENTS.md b/AGENTS.md
index 090e4dbd..e2bfdf38 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -56,13 +56,14 @@ cargo test -p hudi-core --lib --no-default-features
toolchain in front of every consumer, for a tier that only engages when a
merge exceeds
`hoodie.memory.merge.max.size`.
-It is **default-on in `hudi-core`, `hudi-datafusion` and `hudi`**, and the
latter two forward it
-rather than taking `hudi-core` with its defaults: a consumer can only opt out
of a feature its
-direct dependency exposes, and unification through `hudi-datafusion` would
otherwise hand the tier
-back. The opt-out is therefore `default-features = false` on `hudi`, and it
holds with the
-`datafusion` feature on. **The Python wheel and the cxx bridge do not opt
out** — both take `hudi`
-with defaults and build RocksDB. Dropping the feature removes the tier, and a
merge past the budget
-then fails with a `CoreError::Unsupported` naming the config rather than
spilling, which is why the
+It is **default-on in `hudi-core`, `hudi-datafusion`, `hudi` and `hudi-ffi`**,
and the latter
+three forward it rather than taking their dependency with its defaults: a
consumer can only opt out
+of a feature its direct dependency exposes, and unification through
`hudi-datafusion` would
+otherwise hand the tier back. The opt-out is therefore `default-features =
false` on `hudi`, and it
+holds with the `datafusion` feature on; a consumer of `hudi-ffi` makes the
same call there.
+**The Python wheel and the cxx bridge do not opt out** — both take `hudi` with
defaults and build
+RocksDB. Dropping the feature removes the tier, and a merge past the budget
then fails with a
+`CoreError::Unsupported` naming the config rather than spilling, which is why
the
published artifacts keep it.
A test that needs the disk tier must be `#[cfg(feature = "spill-rocksdb")]`;
the
diff --git a/crates/ffi/Cargo.toml b/crates/ffi/Cargo.toml
index c9abb16a..0e218693 100644
--- a/crates/ffi/Cargo.toml
+++ b/crates/ffi/Cargo.toml
@@ -34,11 +34,17 @@ name = "hudi_ffi"
crate-type = ["cdylib", "rlib"]
[features]
+default = ["spill-rocksdb"]
+# Forwarded so a consumer of this crate can drop the merge map's on-disk tier
+# (and with it RocksDB, bindgen and libclang). A consumer can only opt out of a
+# feature its direct dependency exposes, so taking hudi with its defaults here
+# would hand the tier to every consumer with no way to decline it.
+spill-rocksdb = ["hudi/spill-rocksdb"]
# Exports a deliberately panicking function, so a caller can prove a panic does
# not cross the C ABI. Off by default: it must not exist in a shipped library.
ffi-test-hooks = []
[dependencies]
-hudi = { path = "../hudi", version = "0.6.0-dev" }
+hudi = { path = "../hudi", version = "0.6.0-dev", default-features = false }
arrow = { workspace = true, features = ["ffi"] }
tokio = { workspace = true }
