AND query not working on stopwords as expected

2015-02-16 Thread Arun Rangarajan
Solr version 4.2.1 In my schema, I have text type defined as follows: --- fieldType name=text class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.StopFilterFactory words=stopwords.txt

Re: AND query not working on stopwords as expected

2015-02-16 Thread Yonik Seeley
On Mon, Feb 16, 2015 at 4:32 PM, Arun Rangarajan arunrangara...@gmail.com wrote: [...] This query q=name:ofrows=0 gives no results as expected. However, this query: q=name:of AND all_class_ids:(371)rows=0 gives results and is equal to the same number of results as

Re: AND query not working on stopwords as expected

2015-02-16 Thread Erick Erickson
Query parsing is not strict boolean logic, this trips up many people even though AND, NOT and OR are used. See: https://lucidworks.com/blog/why-not-and-or-and-not/ I think what you've really got at the top level is a single MUST clause, namely all_class_ids:(371). What is _not_ happening here is

Re: AND query not working on stopwords as expected

2015-02-16 Thread Alexandre Rafalovitch
On 16 February 2015 at 19:12, Jack Krupansky jack.krupan...@gmail.com wrote: In fact, it would be better to only remove stop words at query time when they are not at either end of the query. And how is that achieved in Solr? This sounds interesting but stretches my knowledge of the available

Re: AND query not working on stopwords as expected

2015-02-16 Thread Jack Krupansky
Notice that I said would be rather than is! Yeah, Solr is basically broken WRT intelligent stop word handling, but nobody wants to admit it. edismax does have some limited support for the case of the query being all stop words, but that doesn't work for more complex queries with operators and the

Re: AND query not working on stopwords as expected

2015-02-16 Thread Alexandre Rafalovitch
Well, there is CommonGrams and CommonGramsQuery filters (e.g. http://www.solr-start.com/javadoc/solr-lucene/org/apache/lucene/analysis/commongrams/CommonGramsQueryFilter.html ). But I haven't seen them in use much. If the description above (about the first/last token) would actually be useful, it

Re: AND query not working on stopwords as expected

2015-02-16 Thread Jack Krupansky
Specifically what is happening is that the query parser passes of to the analyzer for the name field, which removes the stopwords, including of, which results in no term to be queried. A Lucene BooleanQuery with no terms will match... nothing. But then when you add another clause, you have the