OlivierJaquemet opened a new issue, #16443: URL: https://github.com/apache/lucene/issues/16443
### Description ## Disclaimer Exactly like I already did for bug #16441, I found this bug in our codebase and manually wrote the unit tests covering the various cases. I then used Claude AI to help identify the root cause and draft this report. Finally, I verified both the reported behavior and the proposed patch against Lucene's main branch (running all tests) before submitting this report and corresponding merge request. I could not find any policy in the [Contributing to Lucene Guide](https://github.com/apache/lucene/blob/main/CONTRIBUTING.md) preventing me from going this way. My apologies if this is not authorized. ## Summary `MultiFieldQueryParser`'s javadoc states that under `AND_OPERATOR`: > "the result will be: `+(title:term1 body:term1) +(title:term2 body:term2)`... In other words, all the query's terms must appear, but it doesn't matter in what fields they appear." This guarantee only holds when each term reaches the parser as a **separate grammar-level token** (i.e. real whitespace between them, handled by `QueryParser.jj`'s own `Clause()`/`addClause()`/`MultiTerm()` machinery term-by-term). It does not hold when several terms are produced by the **analyzer alone** from a *single* piece of query text passed to one `getFieldQuery(null, text, quoted)` call — for example an escaped separator with no real whitespace, word decompounding, CJK segmentation, or synonym expansion. In that case, the terms are always combined with `Occur.SHOULD`, regardless of `setDefaultOperator(...)`. ## Reproduction ```java Analyzer analyzer = new ClassicAnalyzer(); // splits on `:`, unlike StandardAnalyzer MultiFieldQueryParser parser = new MultiFieldQueryParser(new String[] {"field1", "field2"}, analyzer); parser.setDefaultOperator(QueryParser.Operator.AND); Query q = parser.parse(QueryParser.escape("foo:bar")); System.out.println(q.toString()); ``` **Expected:** ``` +(field1:foo field2:foo) +(field1:bar field2:bar) ``` (both terms required, each as a disjunction across fields — matching the class's documented AND semantics) **Actual:** ``` (field1:foo field2:foo) (field1:bar field2:bar) ``` (no `+` at all — both term positions are optional, i.e. the query behaves as OR) ## Root cause Inside `MultiFieldQueryParser#getFieldQuery(String field, String queryText, boolean quoted)`, when a per-field analysis yields more than one token (`maxTerms > 1`), the method groups the fields for each term position into its own `SHOULD`-combined `BooleanQuery` (correct: fields are alternatives for a given term). It then combines these **per-position groups** using: ```java protected Query getMultiFieldQuery(List<Query> queries) throws ParseException { BooleanQuery.Builder query = newBooleanQuery(); for (Query sub : queries) { query.add(sub, BooleanClause.Occur.SHOULD); // always SHOULD } return query.build(); } ``` `getMultiFieldQuery` is also the method used to combine *fields* for a single term (where `SHOULD` is always correct, since fields are genuine alternatives). It is being reused here to combine *term positions* as well, where the semantics should instead depend on the default operator. There is no operator-aware step anywhere in this code path for the "single call, multiple analyzed terms" case — the class relies entirely on the surrounding grammar (`Clause()`/`addClause()`) to apply the operator between terms, which only works when the terms are separate grammar tokens to begin with. ### Version and environment details Affected version: main and 10.4.0 Component: lucene-queryparser (classic package) Affected classes: `MultiFieldQueryParser#getFieldQuery(String, String, boolean)` / `getMultiFieldQuery` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
