JingsongLi opened a new pull request, #1091: URL: https://github.com/apache/paimon-rust/pull/1091
### Purpose Expose one global-index build API selected by index type, following Java's `GlobalIndexer.create` and `GlobalIndexBuilderUtils`. Sorting is an internal implementation choice; callers should not select separate sorted, full-text, Lumina or vindex table builders. ### Brief change log - Introduce `Table::new_global_index_build_builder()` and `GlobalIndexBuildBuilder` for all existing Rust global-index families: BTree, bitmap, multivalue, FM, full-text, Lumina and the five vindex types. - Share column/type normalization, partition-filter validation and prepare/execute ownership. `build()` returns unpublished commit messages; `execute()` prepares and commits with the planned-snapshot guard. - Make the specialized builders private implementations and migrate Rust callers, examples, DataFusion procedures and the Python binding/stubs. Preserve the Lumina-specific SQL procedure's type restriction. - Use the table's REST-aware snapshot manager for vector preparation; honor per-build shard sizing and partition filters. - Apply Java-compatible external global-index placement and DEIX source metadata to prepared generic indexes so the existing commit conflict checks also protect their inputs. - Retain submitted or returned files on failed or uncertain commits; only failed private preparation cleans its own outputs. ### Tests - `cargo test -p paimon --features fulltext --lib index_build --no-fail-fast`: 142 passed, 2 previously ignored. - `cargo test -p paimon --lib global_index_build_builder`: 54 passed, 1 previously ignored, including the build without the full-text feature. - `cargo test -p paimon-datafusion --features fulltext --test procedures index --no-fail-fast`: 27 passed. - `cargo +1.98.0 clippy -p paimon -p pypaimon_rust --all-targets --features paimon/fulltext -- -D warnings`: passed. - `cargo fmt --all -- --check` and `git diff --check`: passed. - Paired PyPaimon REST tests exercise real prepared full-text and all five vindex formats, both committers, partition conjunction, external placement, source conflicts and Python fast-search reads. Together with existing index-build tests: 153 passed and 19 subtests passed. ### API and Format Replaces the specialized public table factories/types with `GlobalIndexBuildBuilder`; the Python binding is a parameter/commit-message wrapper around core. The shared type-normalization helper is now named `normalize_global_index_type`. Existing index formats and source metadata versions are unchanged. Generic builds now populate the already-supported DEIX source metadata. ### Documentation The new builder documents prepare vs execute, partition conjunction and supported column definitions; the Python stub and benchmark example use the common API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
