JingsongLi opened a new pull request, #1095: URL: https://github.com/apache/paimon-rust/pull/1095
### Purpose Move the local full-text search capabilities already available in PyPaimon into Rust core. The existing Rust builder cannot express partition/data predicates or provide an owned, snapshot-pinned Scan -> Plan -> Read contract for the Python wrapper. ### Brief change log - Add structured `with_query`, filters, and owned `FullTextScan`, `FullTextScanPlan`, and `FullTextRead` types. Preserve the existing plain-text SQL builder API. - Resolve a snapshot once, honor REST/branch/time-travel selection, prune partitions, and preserve the bound read schema across schema-only renames. - Follow Java full-text/scalar FAST, FULL and DETAIL coverage semantics. Refine inexact scalar candidates only when configured; exclude composite scalar indexes from search pre-filters as Java does. - Keep complete indexed/raw corpora for BM25, collect Boolean/boost leaf candidates before final Top-K, and honor `global-index.thread-num`. - Share exact row-ID filtering with vector searches and expose thin Python bindings plus type stubs. Index search, raw reading, scalar evaluation and ranking remain in core. ### Tests - Full-text builder: 24 passed; vector regression: 61 passed; hybrid search: 12 passed. - Clippy 1.98.0, all targets for core and Python binding with full-text enabled: passed. - Rustfmt and minimal-feature core compile: passed. - Built the local Python wheel and ran 106 PyPaimon tests with native plan/read/write/update/commit gates enabled. All passed; actual counters: 113 plans, 105 reads, 10 writes, 1 native commit, and every update category exercised. - REST parity covers structured/boosted queries, Top-K, partial coverage, BTree/bitmap filters, candidate refinement, partitions, typed analyzer options, snapshot selection, rename, deletion vectors, text updates, missing indexes and execution errors. Python scores are compared with the existing reader. ### API and Format Additive core and Python APIs. No storage format change or new dependency. Python integration targets REST Data Evolution tables with global row IDs; PK materialized search remains on its existing path. ### Documentation Document the core builder and snapshot/coverage/filtering semantics in the SQL guide. A paired Apache Paimon Python PR will enable the native local route while keeping CI on Rust main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
