[
https://issues.apache.org/jira/browse/SOLR-8160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18139076#comment-18139076
]
Chris M. Hostetter commented on SOLR-8160:
------------------------------------------
{quote}Let's say it's an analyzed field... then IMO it's _very very_ likely
that the term given will be unchanged if analyzed again. Challenge me on this
if you think otherwise with a realistic config/scenario.
{quote}
Off the top of my head: any use of HTML Char filter (doing "unescaping" or
"decoding" of metacharacters) , or any similar mapping or pattern based char
filter / tokenizer setup that does one way replacements/substitutions?
But also: simple stuff like (Edge)NGrams ... or synonyms? ... anything that
intentionally produces multiple terms at the same position? if you now match
*any* of those variant terms you're not upholding the usecase the parser works
best for: "i have this facet term that has a facet count of X, return those X
documents" ... you're going to start returning X+Z documents based on all those
variant terms (that had their own facet counts)
(and that's before we get to things like "i use PathHierarchyTokenizer when
indexing, and i didn't bother configuring a distinct query analyzer because i
only use the field for facets with the "terms" parser ... which i've definitely
seen in the wild)
But also: what are you imagining this new "analyze" option ("on" or "off" by
default?) should do when the analyzer produces multiple terms that *AREN'T* at
the same position? Most of the time 10000+ long sequence of input strings
produce a really efficient TermInSetQuery, but if one of those input strings
happens to have a space in it you may now get a really inefficient BooleanQuery
wrapping 9999 term queries and one PhraseQuery?
I really feel like we've skipped ahead and are asking the wrong questions ...
why should terms parser do any analysis at all? What usecase does that serve?
Because "terms" as it is (w/o analysis) serves a really useful purpose for
multi-select faceting where we know we don't *need* analysis (and analysis
_may_ break things for many arbitrary field configs) ... and yes i've seen
people intentionally do this on analyzed fields when they want to generate
"word cloud" type results on title/body style fields.
Asking "why doesn't the terms qparser do analysis by default?" seems as weird
to me as asking "why doesn't the field qpaser split on comma by default before
sending each of the resulting strings to the anlayzer?" ... they both currently
serve very straight forward usecases, and either change would break backcompat
in a way that wreck havoc on some people in surprising ways.
if the distinction between the "field" and "term" and "terms" qparsers isn't
clear in the docs, lets make the docs more clear – or let's deprecate the
existing parser names and replace them with longer more explicit names?
> Terms query parser should optionally do query analysis
> -------------------------------------------------------
>
> Key: SOLR-8160
> URL: https://issues.apache.org/jira/browse/SOLR-8160
> Project: Solr
> Issue Type: Improvement
> Components: query parsers, search
> Affects Versions: 5.3
> Reporter: Devansh Dhutia
> Assignee: David Smiley
> Priority: Major
>
> Field setup as
> {code}
> <field name="cs" type="string_ci" indexed="true" stored="true"
> multiValued="false" required="false" />
> <fieldType name="string_ci" class="solr.TextField" sortMissingLast="true">
> <analyzer type="index">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.KeywordTokenizerFactory" />
> <filter class="solr.LowerCaseFilterFactory" />
> </analyzer>
> </fieldType>
> {code}
> Value sent to cs field for indexing include: AA, BB
> Following is observed
> {code}&fq={!terms f=cs}AA,BB{code} yields 0 results
> {code}&fq={!terms f=cs}aa,bb{code} yields 2 results
> {code}&fq=cs:(AA BB){code} yields 2 results
> {code}&fq=cs:(aa bb){code} yields 2 results
> The first variant above should behave like the other 3 & obey query time
> analysis
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
