Gabriel39 commented on code in PR #68815:
URL: https://github.com/apache/doris/pull/68815#discussion_r4229036378
##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/VectorSearchTableValuedFunction.java:
##########
@@ -64,10 +64,12 @@ public class VectorSearchTableValuedFunction extends
LanceExternalSearchTableVal
private static final String NPROBES = "nprobes";
private static final String REFINE_FACTOR = "refine_factor";
private static final String EF = "ef";
+ private static final String DISTANCE_LOWER_BOUND = "distance_lower_bound";
+ private static final String DISTANCE_UPPER_BOUND = "distance_upper_bound";
private static final String USE_INDEX = "use_index";
private static final Set<String> PROPERTIES = ImmutableSet.of(
TABLE, COLUMN, QUERY_VECTOR, TOP_K, OFFSET, METRIC, FILTER,
- NPROBES, REFINE_FACTOR, EF, USE_INDEX, QUERY_PARALLELISM);
+ NPROBES, REFINE_FACTOR, EF, USE_INDEX, QUERY_PARALLELISM,
DISTANCE_LOWER_BOUND, DISTANCE_UPPER_BOUND);
Review Comment:
Fixed in bc67e3bdcd. Fixed by allowing both distance-bound properties in the
prepared TVF parameter path. Added repeated EXECUTE binding coverage for
lower-only, upper-only, both parameters, and mixed literal/parameter bounds,
including invalid FLOAT values, reversed bounds, NULL values, and recovery
after failed execution. The 76 related FE tests and FE Checkstyle pass.
##########
be/src/format_v2/table/lance_reader.cpp:
##########
@@ -1078,6 +1103,17 @@ Status
LanceTableReader::_configure_vector_search(LanceScanner* scanner,
}
}
+ if (vector.__isset.distance_lower_bound ||
vector.__isset.distance_upper_bound) {
+ const float lower = static_cast<float>(vector.distance_lower_bound);
+ const float upper = static_cast<float>(vector.distance_upper_bound);
+ // nearest() clears the range; install bounds only after setting the
query vector.
+ if (lance_scanner_set_distance_range(
+ scanner, vector.__isset.distance_lower_bound ? &lower :
nullptr,
+ vector.__isset.distance_upper_bound ? &upper : nullptr) !=
0) {
Review Comment:
Fixed in bc67e3bdcd. Addressed by avoiding the affected native index path
for single-sided ranges. FE now plans exact fragment scans and reports
DISTANCE_RANGE_FALLBACK, even if use_index=true; it does not synthesize an
endpoint for an omitted bound. Reproduced the pinned native FLT_MAX DOT false
negative locally and added a synthetic IVF_FLAT fixture plus BE coverage with
and without prefiltering. This is a conservative fallback, not a modification
to the pinned Lance dependency; the additional scan cost is documented. All 68
LanceTableReader tests pass on a fresh ASAN build, including these cases; the
76 related FE tests and formatting checks also pass.
##########
be/src/format_v2/table/lance_reader.cpp:
##########
@@ -1177,6 +1202,17 @@ Status
LanceTableReader::_configure_vector_search(LanceScanner* scanner,
}
}
+ if (vector.__isset.distance_lower_bound ||
vector.__isset.distance_upper_bound) {
+ const float lower = static_cast<float>(vector.distance_lower_bound);
+ const float upper = static_cast<float>(vector.distance_upper_bound);
+ // nearest() clears the range; install bounds only after setting the
query vector.
+ if (lance_scanner_set_distance_range(
Review Comment:
Fixed in bc67e3bdcd. Fixed by allowing bounded index plans only for
explicitly identified IVF_FLAT segments with both bounds present. Quantized and
unknown vector index subtypes fall back to exact fragment scans, so approximate
scores cannot eliminate a row before exact evaluation. Unbounded searches
retain their existing index selection. Added planner coverage across all vector
index subtypes and an IVF_SQ fixture reproducing approximate distance 0 versus
exact distance approximately 0.0004. The fallback and its scan cost are
documented. All 68 LanceTableReader tests pass on a fresh ASAN build, including
these cases; the 76 related FE tests and formatting checks also pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
