JingsongLi opened a new pull request, #10482: URL: https://github.com/apache/paimon/pull/10482
## Purpose Depends on https://github.com/apache/paimon-rust/pull/1088. Draft until the core API is available on Rust main. Use Rust for PyPaimon's existing local BTree and Bitmap global-index builds on eligible REST Data Evolution tables. Snapshot selection, range planning, reading, sorting and file generation stay in Rust core; Python retains the existing GlobalIndexBuilder and CommitMessage API. ## Changes - Dispatch local sorted-index builds when write.native.enabled is enabled and the table/binding is eligible. - Pass the existing partition filter and user options through the native builder and reuse the commit-message serializer bridge. - Keep authorization checks on the existing Python build path. Native build errors propagate once a build starts. - Read Java-compatible DataEvolutionIndexSourceMeta and validate staged index sources with the Python committer too: reject target-column changes, deleted column deltas, missing/future snapshots and removed row ranges. Pure appends and unrelated column updates remain valid. - Preserve prepared files after commit exceptions, including a successfully published snapshot whose response is lost. Explicit TableCommit.abort remains the caller's choice. - Add REST integration coverage for BTree/Bitmap, partitions and incremental builds, external paths, typed keys, both committers, NULL/empty-set partition predicates, authorization and failure paths. ## Validation - PYPAIMON_TEST_NATIVE_PLAN=1 pytest native_sorted_index_build_test.py native_vector_search_test.py global_index_build_test.py write/conflict_detection_test.py file_store_commit_test.py: 283 passed, 32 subtests; 104 native plans exercised - Final native_sorted_index_build_test.py: 87 passed - Flake8 with dev/cfg.ini and Python 3.6 syntax checks passed CI continues to install paimon-rust from main. Merge the Rust counterpart first so these native tests exercise the new API on CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
