airborne12 commented on code in PR #69791:
URL: https://github.com/apache/doris/pull/69791#discussion_r4449691621
##########
be/src/storage/predicate_collector.cpp:
##########
@@ -300,7 +296,12 @@ Status MatchPredicateCollector::collect(RuntimeState*
state, const TabletSchemaS
options.timezone = &state->timezone_obj();
const std::string value = right_literal->value(options);
std::vector<TermInfo> term_infos;
- if (match_analyzer_ctx != nullptr) {
+ if (!InvertedIndexAnalyzer::should_analyzer(index_meta->properties()))
{
+ // A keyword index stores the whole value as one term, case and
spaces included, and
+ // its reader looks it up unanalyzed: tag MATCH 'Hello World'
scores the single term
+ // "Hello World", as SEARCH('tag:"Hello World"') does.
+ term_infos.emplace_back(value);
Review Comment:
Known limitation on this branch, not fixed here. Statistics on branch-4.1
are keyed per field, so a keyword index and an analyzed index of one column
share one entry when both are scored in the same query, and at least one clause
can get the wrong idf. It takes two indexes on one column, both selected in one
query with `USING ANALYZER`; single-index queries are not affected.
Master fixed this separately in #69901 by keying MATCH statistics per index.
That does not carry over to this branch directly, because its reader path
differs, so it would be its own backport rather than part of this bug fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
