Oh, I see, haven't noticed you used solr 4.0. luke can only read 3.5 at most, at the moment. So when you search with a leading wildcard, do both your app and the SOLR admin search give the same results?
Probably you can show relevant parts of your schema and solrconfig? Like type(s) definition from schema and searchers from solrconfig. Do you have any custom query parser implementations or any other custom components? On Thu, Jan 19, 2012 at 9:26 AM, Shyam Bhaskaran < shyam.bhaska...@synopsys.com> wrote: > Dimitry, > > I have used lukeall-3.5.0.jar and when trying to open the index it gives > me the error "No Valid Directory at the location, try another location" > > When using the below command I see this error "luke > java.lang.ArrayIndexOutOfBoundsException: 1" > java -cp C:\lukeall-3.5.0.jar org.getopt.luke.Luke -index > C:\solr\home\data\docs_index\index\ > > We are using Solr 4.0 > > -Shyam > > -----Original Message----- > From: Shyam Bhaskaran [mailto:shyam.bhaska...@synopsys.com] > Sent: Thursday, January 19, 2012 11:49 AM > To: solr-user@lucene.apache.org > Subject: RE: Question on Reverse Indexing > > Dimitry, > > I downloaded Luke but it was not working for me against solr indexes. > > But using the solr analysis page I did not find any reversed sequences on > the field. > > -Shyam > > > -----Original Message----- > From: Shyam Bhaskaran [mailto:shyam.bhaska...@synopsys.com] > Sent: Thursday, January 19, 2012 6:29 AM > To: solr-user@lucene.apache.org > Subject: RE: Question on Reverse Indexing > > Dimitry, > > Completed a clean index and I still see the same behavior. > > Did not use Luke but from the search page we use leading wild card search > is working. > > -Shyam > > -----Original Message----- > From: Dmitry Kan [mailto:dmitry....@gmail.com] > Sent: Wednesday, January 18, 2012 5:07 PM > To: solr-user@lucene.apache.org > Subject: Re: Question on Reverse Indexing > > Shyam, > > You still didn't say if you have started re-indexing from the clean index, > i.e. if you have removed all the data prior to re-indexing. > You can use the luke (http://code.google.com/p/luke/) to check the > contents > of your text field, and see if it still contains reversed sequences. > > On Wed, Jan 18, 2012 at 1:09 PM, Shyam Bhaskaran < > shyam.bhaska...@synopsys.com> wrote: > > > Dimitry, > > > > We are using Solr 4.0. To confirm server caching issues I have restarted > > our tomcat webserver after performing a re-index. > > > > For reverseIndexing we have defined a fieldType "text_rev" and this > > fieldyType was used against the fields. > > > > <fieldType name="text_rev" class="solr.TextField" sortMissingLast="true" > > omitNorms="true"> > > <analyzer type="index"> > > <tokenizer > > class="com.es.solr.backend.analysis.standard.SolvNetTokenizerFactory"/> > > <filter class="solr.StopFilterFactory" > > words="stopwords.txt" ignoreCase="true"/> > > <filter > > class="com.es.solr.backend.analysis.standard.SolvNetFilterFactory"/> > > <filter class="solr.SynonymFilterFactory" > > synonyms="synonyms.txt" ignoreCase="true" expand="true"/> > > <filter > > > class="com.es.solr.backend.analysis.standard.SpecialCharSynonymFilterFactory"/> > > <filter class="solr.LowerCaseFilterFactory"/> > > <filter class="solr.ReversedWildcardFilterFactory" > > withOriginal="true" > > maxPosAsterisk="3" maxPosQuestion="2" > > maxFractionAsterisk="0.33"/> > > </analyzer> > > <analyzer type="query"> > > <tokenizer > > class="com.es.solr.backend.analysis.standard.SolvNetTokenizerFactory"/> > > <filter class="solr.StopFilterFactory" > > words="stopwords.txt" ignoreCase="true"/> > > <filter > > class="com.es.solr.backend.analysis.standard.SolvNetFilterFactory"/> > > <filter class="solr.LowerCaseFilterFactory"/> > > <filter class="solr.StopFilterFactory" > > words="stopwords.txt" ignoreCase="true"/> > > </analyzer> > > </fieldType> > > > > But when it was found that ReversedWildcardFilterFactory is adding > > performance burden we removed the ReversedWildcardFilterFactory filter > > <filter class="solr.ReversedWildcardFilterFactory" > > withOriginal="true" > > maxPosAsterisk="3" maxPosQuestion="2" > > maxFractionAsterisk="0.33"/> > > and the whole collection was re-indexed. > > > > But even after removing the ReversedWildcardFilterFactory leading wild > > card search like *lock is working. > > > > -Shyam > > > > -----Original Message----- > > From: Dmitry Kan [mailto:dmitry....@gmail.com] > > Sent: Wednesday, January 18, 2012 4:26 PM > > To: solr-user@lucene.apache.org > > Subject: Re: Question on Reverse Indexing > > > > OK. Not sure what is your system architecture there, but could your > queries > > stay cached in some server caches even after you have re-indexed your > data? > > The way the index level leading wildcard works (reading SOLR 3.4 code, > but > > seems to be true circa 1.4) is that the following check is done for the > > analysis chain: > > > > [code src=SolrQueryParser.java] > > boolean allow = false; > > ... > > if (factory instanceof ReversedWildcardFilterFactory) { > > allow = true; > > ... > > } > > ... > > if (allow) { > > setAllowLeadingWildcard(true); > > } > > [/code] > > > > so practically what you described can happen if > > the ReversedWildcardFilterFactory is still mentioned in one of your > shards. > > A weird question, but have you reindexed your data to a clean index or on > > top of the existing one? > > > > On Wed, Jan 18, 2012 at 12:35 PM, Shyam Bhaskaran < > > shyam.bhaska...@synopsys.com> wrote: > > > > > Dimitry, > > > > > > Using http://localhost:7070/solr/docs/admin/analysis.jsp passed the > > query > > > *lock and did not find ReversedWildcardFilterFactory to the indexer or > > any > > > other filters that could do the reversing. > > > > > > -Shyam > > > > > > -----Original Message----- > > > From: Dmitry Kan [mailto:dmitry....@gmail.com] > > > Sent: Wednesday, January 18, 2012 2:26 PM > > > To: solr-user@lucene.apache.org > > > Subject: Re: Question on Reverse Indexing > > > > > > Just to play safe here, can you double check that the reversing is not > > any > > > more the case by issuing a query through the admin analysis page? > > > > > > Dmitry > > > > > > On Wed, Jan 18, 2012 at 4:23 AM, Shyam Bhaskaran < > > > shyam.bhaska...@synopsys.com> wrote: > > > > > > > Hi Francois, > > > > > > > > I understand that disabling of ReversedWildcardFilterFactory has > > improved > > > > the performance. > > > > > > > > But I am puzzled over how the leading wild card search like *lock is > > > > working even though I have now disabled the > > ReversedWildcardFilterFactory > > > > and the indexes have been created without ReversedWildcardFilter ? > > > > > > > > How does reverse indexing work even after disabling > > > > ReversedWildcardFilterFactory? > > > > > > > > Can anyone explain me how this feature is working. > > > > > > > > -Shyam > > > > > > > > -----Original Message----- > > > > From: François Schiettecatte [mailto:fschietteca...@gmail.com] > > > > Sent: Wednesday, January 18, 2012 7:49 AM > > > > To: solr-user@lucene.apache.org > > > > Subject: Re: Question on Reverse Indexing > > > > > > > > Using ReversedWildcardFilterFactory will double the size of your > > > > dictionary (more or less), maybe the drop in performance that you are > > > > seeing is a result of that? > > > > > > > > François > > > > > > > > On Jan 17, 2012, at 9:01 PM, Shyam Bhaskaran wrote: > > > > > > > > > Hi, > > > > > > > > > > For reverse indexing we are using the ReversedWildcardFilterFactory > > on > > > > Solr 4.0 > > > > > > > > > > > > > > > <filter class="solr.ReversedWildcardFilterFactory" > > withOriginal="true" > > > > > > > > > > maxPosAsterisk="3" maxPosQuestion="2" maxFractionAsterisk="0.33"/> > > > > > > > > > > > > > > > ReversedWildcardFilterFactory was helping us to perform leading > wild > > > > card searches like *lock. > > > > > > > > > > But it was observed that the performance of the searches was not > good > > > > after introducing ReversedWildcardFilterFactory filter. > > > > > > > > > > Hence we disabled ReversedWildcardFilterFactory filter and > re-created > > > > the indexes and this time we found the performance of Solr query to > be > > > > faster. > > > > > > > > > > But surprisingly it is observed that leading wild card searches > were > > > > still working inspite of disabling the ReversedWildcardFilterFactory > > > filter. > > > > > > > > > > > > > > > This behavior is puzzling everyone and wanted to know how this > > behavior > > > > of reverse indexing works? > > > > > > > > > > Can anyone share with me on this Solr behavior. > > > > > > > > > > -Shyam > > > > > > > > > > > > > > > > > > > > > > -- > > > Regards, > > > > > > Dmitry Kan > > > > > > > > > > > -- > > Regards, > > > > Dmitry Kan > > > > > > -- > Regards, > > Dmitry Kan > -- Regards, Dmitry Kan
