github-actions[bot] commented on code in PR #66861:
URL: https://github.com/apache/doris/pull/66861#discussion_r3800594371
##########
be/src/storage/index/snii/query/internal/wildcard_matcher.h:
##########
@@ -43,37 +45,76 @@ namespace doris::snii::query::internal {
template <class Alloc = std::allocator<uint8_t>>
class WildcardMatcher {
public:
- explicit WildcardMatcher(std::string_view pattern) : pattern_(pattern) {}
+ explicit WildcardMatcher(std::string_view pattern)
+ : pattern_(pattern), pattern_valid_(is_valid_utf8(pattern)) {}
bool operator()(std::string_view text) {
+ if (!pattern_valid_ || !is_valid_utf8(text)) {
Review Comment:
[P1] Preserve wildcard matches for raw keyword terms
SNII's non-analyzed writer persists raw `VARCHAR` bytes without UTF-8
validation, so a legal value such as `cast(unhex('61FF') as string)` creates a
term containing bytes `0x61 0xff` that reaches this guard. Returning `false`
makes `body:a*` and `body:**` silently omit that row, even though the old byte
matcher returned it and `body:*` still returns it through `MATCH_ALL_DOCS`.
This changes results for existing indexes and makes equivalent wildcard forms
disagree. Please preserve compatible handling where possible, or propagate a
non-OK/bypass result for malformed dictionary terms instead of treating them as
clean non-matches; add an end-to-end SNII keyword case covering these three
patterns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
