Kind of a shot in the dark here, but your parameters for index and query on WordDelimiterFilterFactory are different, especially suspicious is catenateWords.
You could test this by looking in your index with the SOLR admin page and/or Luke to see what your actual terms are..... And don't forget you'll have to re-index after restarting SOLR for any index changes to take effect.... HTH Erick On Thu, Mar 11, 2010 at 2:20 PM, Ya-Wen Hsu <y...@eline.com> wrote: > Yonik, thank you for your reply. When I don't use PreserveOriginal = 1 for > WordDelimiterFilterFactory, the query "ain't" is parsed as "ain t" and no > match is found in this case too. If I remove ' from the query, then I can > get results. I used the analysis tool and see the term ain't is processed as > "ain t", and get matches when the title includes "ain't". But I got no > result when using ain't query with dismax. > > The debug output looks like: > (NON-MATCH) Failure to meet condition(s) of required/prohibited clause(s) > +(long_description:"ain t"^2.0 | name:"ain t"^3.0 | search_keywords:"ain > t")~0.1 (long_description:save^2.0 | name:save^3.0 | > search_keywords:saved)~0.1) () > > > Below is my configuration for text field type. > > <fieldType name="text" class="solr.TextField" positionIncrementGap="100"> > <analyzer type="index"> > <tokenizer class="solr.WhitespaceTokenizerFactory"/> > <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" > ignoreCase="true" expand="false"/> > <filter class="solr.StopFilterFactory" ignoreCase="true" > words="stopwords.txt" enablePositionIncrements="true" /> > <filter class="solr.WordDelimiterFilterFactory" > generateWordParts="1" generateNumberParts="1" catenateWords="1" > catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"/> > <filter class="solr.LowerCaseFilterFactory"/> > <filter class="solr.EnglishPorterFilterFactory" > protected="protwords.txt"/> > <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> > </analyzer> > <analyzer type="query"> > <tokenizer class="solr.WhitespaceTokenizerFactory"/> > <!--<filter class="solr.SynonymFilterFactory" > synonyms="synonyms.txt" ignoreCase="true" expand="true"/>--> > <filter class="solr.StopFilterFactory" ignoreCase="true" > words="stopwords.txt"/> > <filter class="solr.WordDelimiterFilterFactory" > generateWordParts="1" generateNumberParts="1" catenateWords="0" > catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/> > <filter class="solr.LowerCaseFilterFactory"/> > <filter class="solr.EnglishPorterFilterFactory" > protected="protwords.txt"/> > <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> > </analyzer> > </fieldType> > > > I get results back when I tried to use solr.LowerCaseTokenizerFactory > instead of solr.WhitespaceTokenizerFactory. However, the concern here is > this might reduce the quality of relevant search. Does anyone have a better > idea on what to try next? Thanks! > > Wen > -----Original Message----- > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik > Seeley > Sent: Thursday, March 11, 2010 10:51 AM > To: solr-user@lucene.apache.org > Subject: Re: dismax and WordDelimiterFilterFactory with PreserveOriginal = > 1 > > On Thu, Mar 11, 2010 at 1:07 PM, Ya-Wen Hsu <y...@eline.com> wrote: > > Hi all, > > > > I'm facing the same issue as previous post here: > http://www.mail-archive.com/solr-user@lucene.apache.org/msg19511.html. > Since no one answers this post, I thought I'll ask again. In my case, I use > below setting for index > > <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" > generateNumberParts="1" catenateWords="1" catenateNumbers="1" > catenateAll="0" splitOnCaseChange="0" preserveOriginal="1"/> > > and > > <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" > generateNumberParts="1" catenateWords="0" catenateNumbers="0" > catenateAll="0" splitOnCaseChange="0" preserveOriginal="1"/> for query. > > > > When I use query with word "ain't", no result is returned. When I turned > on the logging, I found the word is interpreted as "(ain't ain) t". > > > The problem is preserving the original in the query analyzer - try > removing that. And if you aren't doing prefix or wildcard queries, > preserveOriginal doesn't buy you anything but wasted index space. > > It's the same issue of why you can't generate and catenate at the same > time with the query parser. > > -Yonik > http://www.lucidimagination.com >
