SteNicholas opened a new issue, #454: URL: https://github.com/apache/paimon-cpp/issues/454
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-cpp/issues) and found nothing similar. ### Paimon-cpp version `main` at commit `50ef548ec87cc444a153fb2a03789f9be768d610`, with #448 applied. ### Minimal reproduce step 1. Create a data-evolution table with row tracking and `deletion-vectors.enabled = true`. Write 10 rows with a vector column, then build a vector global index on that column. 2. Delete the three rows nearest to a query vector, using deletion vectors. 3. Search for that vector with `limit = 3` through `GlobalIndexScan::CreateReader` → `VisitVectorSearch`. 4. Pass the result to `ScanContextBuilder::SetGlobalIndexResult`, then scan and read. ### What doesn't meet your expectations? The index still ranks the deleted rows, so they take all three top-k slots. The read then drops them, because deletion vectors are applied only at read time. The query returns 0 rows instead of 3. #448 (for #403) excludes deleted rows only from full-text ranking: `LiveRowGlobalIndexReader::VisitVectorSearch` delegates unchanged, and live rows are loaded only before a full-text search fans out to the shards. Java filters vector search with the same live rows (apache/paimon#8459). **Expected:** `limit = k` returns k live rows whenever at least k live rows pass the search's `pre_filter` and `predicate`. ### Anything else? Follow-up of #403 / #448. Suggestions: - `VectorSearch` already carries a `pre_filter` over global row ids, so `LiveRowGlobalIndexReader::VisitVectorSearch` can combine it with the live rows of the shard through `ReplacePreFilter`, reusing `GlobalIndexLiveRowFilter::LiveRows`, and pass the search on unchanged when the whole shard is live. - Load live rows on the visiting thread before a vector search fans out to the shards, as `LiveRowUnionGlobalIndexReader` does for full-text search. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
