JingsongLi commented on code in PR #1145:
URL: https://github.com/apache/paimon-rust/pull/1145#discussion_r4386455831
##########
crates/paimon/src/table/global_index_scanner/evaluator.rs:
##########
@@ -748,7 +761,7 @@ impl GlobalIndexScanner {
Ok(Some((
crate::table::merge_row_ranges(all_ranges),
coverage(),
- fully_evaluated,
+ fully_evaluated && !(bitmap_retained && bitmap_superset_ops),
Review Comment:
[P2] Scope Bitmap exactness downgrades to the vector-covered domain
This flag also counts Bitmap candidates outside the range the vector index
will rank. In FULL/DETAIL mode, Raw splits carry their scalar indexes, which
are combined with the Index split's scalar entries. `scalar_matched_row_ids`
obtains this global exactness flag before intersecting the returned ranges with
`vector_ranges`. Consequently, a Bitmap candidate removed by that intersection
can still make the remaining exact BTree hits inexact, and
`global-index.filter.refine-from-data=false` then discards them.
I reproduced this through public write/index/search APIs: partition by `id`,
write `id=0, f=2.0` and build BTree(f) plus IVF-flat(embedding); append `id=1,
f=0.0` and build Bitmap(f) only for that partition, without a vector index.
Top-1 with `f > 1.5` returns `[]` in FULL/DETAIL with refinement disabled,
instead of `[0]`. The Bitmap covers `[1,1]`, entirely outside the vector
index's `[0,0]` domain. FAST with refinement disabled and all
refinement-enabled controls return `[0]`. The identical test passes all six
combinations on base `370428b0` and fails the two FULL/DETAIL cases on head
`1f119c29`.
Please determine exactness within the actual vector-covered ranges, or
preserve per-range inexactness through the intersection, and add this
partial-vector-coverage regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
