github-actions[bot] commented on code in PR #69460:
URL: https://github.com/apache/doris/pull/69460#discussion_r4355688361
##########
fe/fe-catalog/src/main/antlr4/org/apache/doris/analysis/SearchLexer.g4:
##########
@@ -42,9 +42,12 @@ fragment QUOTED_CHAR
// ============== Default lexer rules ==============
-AND : 'AND' ;
-OR : 'OR' ;
-NOT : 'NOT' | '!' ;
+// Boolean operators are case-insensitive, matching the query string syntax
used by
+// Lucene/Elasticsearch: `a:x or b:y` is the same as `a:x OR b:y`. The parser
also accepts
+// these tokens in value position, so `field:and` keeps searching for the term
"and".
+AND : [Aa][Nn][Dd] ;
+OR : [Oo][Rr] ;
Review Comment:
[P2] Keep boolean keywords usable as field identifiers
The lexer change also reserves lowercase/mixed-case keywords before a colon,
but `fieldSegment` still accepts only TERM or QUOTED. For example,
`SEARCH('or:foo')` previously parsed as a term on the valid column `or`; it now
starts with an OR token and fails before field binding. The same regression
affects `and:foo`, `not:foo`, and mixed-case spellings in standard, Lucene, and
multi-field modes. Adding these tokens only to fieldValue repairs values after
a colon, so identifier positions also need to accept them without requiring
existing callers to quote their field names.
##########
fe/fe-catalog/src/main/antlr4/org/apache/doris/analysis/SearchLexer.g4:
##########
@@ -94,6 +97,13 @@ LIST_WS : [ \t\r\n\u3000]+ -> skip ;
mode STRING_MODE;
+// A quoted section is tokenized as one unit so that parentheses inside the
quotes do not
+// terminate the clause early, e.g. ANY("All Quiet on the Western Front (2022
film)").
+// It is a separate rule (instead of an alternative inside STRING_CONTENT)
because a negated
+// set inside a closure lets the lexer run past the closing parenthesis when
another ANY/ALL
+// appears later in the same DSL.
+STRING_QUOTED : '"' (ESCAPED_CHAR | ~["\\])* '"' ;
Review Comment:
[P2] Keep raw EXACT quotes from consuming following clauses
EXACT preserves quotes as term data in both visitors, so
`SEARCH('ch:exact(") OR ch:exact(")')` previously built two EXACT leaves
searching for a literal `"`. Here STRING_QUOTED matches the entire `") OR
ch:exact("` and wins over STRING_CONTENT; only the last `)` closes the clause.
Both modes now silently build one EXACT term containing the OR and second
clause, so a keyword-index row holding `"` stops matching. Preserve EXACT's raw
argument boundaries when adding quote-aware lexing, and add a cross-clause
literal-quote regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
