JingsongLi opened a new pull request, #1094: URL: https://github.com/apache/paimon-rust/pull/1094
## What changes PyPaimon's local full-text and vindex builders already support `global-index.build.parallelism`. Native builds currently validate this setting but prepare shards sequentially. - Add the option to core options with the existing default of 1. - Share a bounded shard scheduler between full-text and all five vindex builders. - Keep one reader/writer per shard, the selected snapshot, logical ranges and relative row IDs, following Java's `GenericIndexTopoBuilder.BuildIndexOperator` ownership model. - Return messages and vector timing results in shard plan order, including skipped empty shards. - Stop admission on the first observed error and drain started futures before cleaning private preparation outputs. Preserve the original error and retain files already handed to callers or submitted for commit. No new dependency or public builder API is required. Shard concurrency and each writer's native worker threads are separate controls; the default remains sequential. ## Tests - Deterministic scheduler gates check bounded admission, actual overlap, out-of-order completion, sequential behavior, empty outputs, error/panic draining, original error causes, and private/external output ownership. - Real writers compare serial and parallel metadata and deterministic vector bytes for all six families, query the resulting indexes, exercise incremental coverage, and retain caller outputs when another preparation fails. - `cargo test -p paimon --features fulltext --lib index_build --no-fail-fast`: 159 passed, 2 existing ignored. - `cargo test --locked -p paimon --lib global_index_build --no-fail-fast`: 63 passed, 1 existing ignored. - `cargo +1.98.0 clippy --locked -p paimon -p pypaimon_rust --all-targets --features paimon/fulltext -- -D warnings`: passed. - Rust formatting and whitespace checks: passed. - Built the Python extension and tested it with the accompanying PyPaimon changes: 253 REST/native index tests passed; 94 targeted tests with every Native CI switch enabled passed. Both Python and native committers consumed the prepared files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
