leaves12138 commented on code in PR #1087:
URL: https://github.com/apache/paimon-rust/pull/1087#discussion_r4231693797
##########
crates/paimon/src/table/de_vector_read.rs:
##########
@@ -736,7 +914,7 @@ async fn evaluate_batch_vector_search(
refine = refine_start.map_or(Duration::ZERO, |start| start.elapsed());
}
- if search_mode != GlobalIndexSearchMode::Fast {
+ if route_indexed_to_raw || search_mode != GlobalIndexSearchMode::Fast {
Review Comment:
[P1] Include scalar-unindexed ranges in raw vector scoring
This raw fallback is still entered only for an unevaluable scalar predicate
or a non-FAST vector mode, and its ranges are calculated only from vector
coverage. An evaluable but partially covered scalar index returns `Some(empty)`
for a matching row outside its coverage, so `route_indexed_to_raw` stays false.
If the vector index already covers that row, FULL/DETAIL also have no
vector-unindexed range to recover it.
Real REST reproduction using the companion `vector_table` fixture: (1)
create a BTREE index on `id` over the initial six rows; (2) append `{id: 7,
embedding: [3, 3], pt: 1}`; (3) create an IVF-flat index on `embedding` with
dimension 2 and nlist 1, now covering all seven rows; (4) search `[0, 0]` with
`id = 7`, limit 10, and `scalar-index.search-mode=full` or `detail`. Classic
returns 1 hit; both the direct Rust binding and Native PyPaimon return 0
(batch: `[1, 1]` versus `[0, 0]`). All three vector modes reproduce this.
Scalar FAST controls correctly return no hit on both paths. Python search was
patched to raise during the Native calls, so fallback cannot mask the
reproduction.
Java explicitly adds scalar-unindexed ranges using the configured scalar
search mode in `DataEvolutionVectorScan` lines 174-191, intersecting them with
vector coverage when vector mode is FAST. Please preserve that rule here:
compute predicate-field scalar coverage, union its permitted uncovered ranges
into raw scoring even when scalar evaluation succeeds, and apply the residual
predicate there. In FAST vector mode, restrict only these scalar fallback
ranges to vector-covered rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
