[
https://issues.apache.org/jira/browse/SOLR-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358208#comment-15358208
]
Steve Rowe edited comment on SOLR-9185 at 7/1/16 1:44 AM:
----------------------------------------------------------
New patch, switches back to ignoring whitespace (along with comments).
Added new LuceneQParser param {{sow}} (*S*​plit *O*​n
*W*​hitespace) to control whether to split on whitespace. Defaults to
{{SolrQueryParser.DEFAULT_SPLIT_ON_WHITESPACE}} (true).
All Solr core tests pass (with existing split-on-whitespace behavior preserved
as the default), and I've added a couple basic multi-word synonym tests. Needs
more tests to ensure multiword analysis is properly interrupted in the presence
of operators.
was (Author: steve_rowe):
New patch, switches back to ignoring whitespace (along with comments).
Added new LuceneQParser param {{sow}} (*S*plit *O*n *W*hitespace) to control
whether to split on whitespace. Defaults to
{{SolrQueryParser.DEFAULT_SPLIT_ON_WHITESPACE}} (true).
All Solr core tests pass (with existing split-on-whitespace behavior preserved
as the default), and I've added a couple basic multi-word synonym tests. Needs
more tests to ensure multiword analysis is properly interrupted in the presence
of operators.
> Solr's "Lucene"/standard query parser should not split on whitespace before
> sending terms to analysis
> -----------------------------------------------------------------------------------------------------
>
> Key: SOLR-9185
> URL: https://issues.apache.org/jira/browse/SOLR-9185
> Project: Solr
> Issue Type: Bug
> Reporter: Steve Rowe
> Assignee: Steve Rowe
> Attachments: SOLR-9185.patch, SOLR-9185.patch, SOLR-9185.patch
>
>
> Copied from LUCENE-2605:
> The queryparser parses input on whitespace, and sends each whitespace
> separated term to its own independent token stream.
> This breaks the following at query-time, because they can't see across
> whitespace boundaries:
> n-gram analysis
> shingles
> synonyms (especially multi-word for whitespace-separated languages)
> languages where a 'word' can contain whitespace (e.g. vietnamese)
> Its also rather unexpected, as users think their
> charfilters/tokenizers/tokenfilters will do the same thing at index and
> querytime, but
> in many cases they can't. Instead, preferably the queryparser would parse
> around only real 'operators'.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
