JingsongLi opened a new pull request, #10494: URL: https://github.com/apache/paimon/pull/10494
### Purpose Delegate existing local full-text searches to Rust core for eligible REST Data Evolution tables with row tracking and Parquet files. Query parsing, snapshot/index planning, scalar filtering, raw scans and ranking run in core; Python forwards builder configuration and wraps scored results. Depends on https://github.com/apache/paimon-rust/pull/1095. This PR is draft until that dependency reaches Rust main. CI keeps installing Rust main; no fork/branch pin is introduced. - Reuse the existing Python builder/result API and full-text analyzer option normalization. - Qualify support before searching. Tables outside the route or predicates not representable by the typed FFI use the Python reader. Once native execution starts, errors propagate without a second search. - Align the classic Python reader with current Java scalar FAST/FULL/DETAIL coverage and configurable candidate refinement, instead of forcing every scalar predicate through FULL reads. - Pin classic read plans, attach scalar files to indexed splits, and require an existing full-text definition before raw search. ### Tests - 106 passed with native plan/read/write/update/commit CI gates enabled against a wheel built from the paired Rust change. - Actual native counters: 113 plans, 105 reads, 10 writes, 1 commit; row-ID, grouped, predicate, upsert, incremental and merge update categories all exercised. - 98 new REST tests compare native/classic result IDs and scores, including all full-text/scalar modes, BTree/bitmap and partial coverage, Boolean/boost queries, pre-Top-K filters, candidate refinement, typed stop words, partition predicates, pinned/empty/error REST snapshots, schema-only renames, deletion vectors, text update invalidation, invalid limits and query authorization. - Eligible-route tests prohibit Python full-text scanning. Missing-index and REST execution failures verify that native errors are not retried through Python. - Existing search reader tests: 79 passed and 19 subtests passed. - Flake8, Python 3.6 source syntax and diff whitespace checks: passed. PK and distributed full-text searches retain their existing path. No new index type or storage format is added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
