Re: spell suggestions help
Be sure to use the Solr Admin UI Analysis page to verify what is happening at each stage of analysis. For BOTH "index" and "query". You only showed us your "query" analyzer... show us the "index" analyzer as well. Did you make sure to delete the index data and completely reindex after changing the "index" analyzer? Or maybe your "index" and "query" analyzers are not in-sync and compatible. Do you have anything in your stopwords file? "and" is usually considered a stop word - so the stop filter would remove it. -- Jack Krupansky -Original Message- From: Rohan Thakur Sent: Friday, April 12, 2013 2:12 AM To: solr-user@lucene.apache.org Subject: Re: spell suggestions help hi jack I am using whitespace toknizer only and before this im using pattern replace to replace & with and but its not working I guess. my query analyser: On Thu, Apr 11, 2013 at 6:03 PM, Jack Krupansky wrote: Try replacing standard tokenizer with whitespace tokenizer in your field types. And make sure not to use any other token filters that might discard special characters (or provide a character map if they support one.) Also, be side to try your test terms in the Solr Admin UI ANalyzer page to see that the "&" is preserved or which stage in term analysis it gets discarded. -- Jack Krupansky -Original Message- From: Rohan Thakur Sent: Thursday, April 11, 2013 7:39 AM To: solr-user@lucene.apache.org Subject: Re: spell suggestions help urlencode replaces & with space thus resulting in results that contains even the single terms like in the case of mobile & accessories it replaces it with mobile accessories and results the document containing even accessories which i dont want. how to tackle this I tried using pattern replace filter at query time to replace & with and but it did not worked I used & => replace with "and" in this but did not worked any guess our help.. thanks regards rohan On Thu, Apr 11, 2013 at 4:39 PM, Rohan Thakur wrote: hi erick do we have to do urlencoding from the php side or does solr supports urlencode? On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson **wrote: Try URL encoding it and/or escaping the & On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur wrote: > hi all > > one thing I wanted to clear is for every other query I have got > correct > suggestions but these 2 cases I am not getting what suppose to be the > suggestions: > > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed > in direct solr spell cheker..but when I query for cattle I get cable > as > only suggestion and not kettle why is this happening i want to get kettle > in suggestion as well im using jarowinkler distance according to which > score for cattle => cable which is coming out to be 0.857 and for cattle => > kettle which is coming out to be 0.777 kettle should also come in > suggestions but its not how can I correct this any one. > > 2) how to query for sentence like "hand blandar & chopper" as & is > delimiter for solr query and thus this query is returning error. > > thanks in advance > regards > Rohan
Re: spell suggestions help
hi jack I am using whitespace toknizer only and before this im using pattern replace to replace & with and but its not working I guess. my query analyser: wrote: > Try replacing standard tokenizer with whitespace tokenizer in your field > types. And make sure not to use any other token filters that might discard > special characters (or provide a character map if they support one.) > > Also, be side to try your test terms in the Solr Admin UI ANalyzer page to > see that the "&" is preserved or which stage in term analysis it gets > discarded. > > -- Jack Krupansky > > -Original Message- From: Rohan Thakur > Sent: Thursday, April 11, 2013 7:39 AM > To: solr-user@lucene.apache.org > Subject: Re: spell suggestions help > > > urlencode replaces & with space thus resulting in results that contains > even the single terms like in the case of mobile & accessories it replaces > it with mobile accessories and results the document containing even > accessories which i dont want. how to tackle this I tried using pattern > replace filter at query time to replace & with and but it did not worked I > used & => replace with "and" in this but did not worked any guess our > help.. > > thanks > regards > rohan > > > On Thu, Apr 11, 2013 at 4:39 PM, Rohan Thakur > wrote: > > hi erick >> >> do we have to do urlencoding from the php side or does solr supports >> urlencode? >> >> >> On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson >> **wrote: >> >> Try URL encoding it and/or escaping the & >>> >>> On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur >>> wrote: >>> > hi all >>> > >>> > one thing I wanted to clear is for every other query I have got correct >>> > suggestions but these 2 cases I am not getting what suppose to be the >>> > suggestions: >>> > >>> > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word >>> indexed >>> > in direct solr spell cheker..but when I query for cattle I get cable as >>> > only suggestion and not kettle why is this happening i want to get >>> kettle >>> > in suggestion as well im using jarowinkler distance according to which >>> > score for cattle => cable which is coming out to be 0.857 and for >>> cattle => >>> > kettle which is coming out to be 0.777 kettle should also come in >>> > suggestions but its not how can I correct this any one. >>> > >>> > 2) how to query for sentence like "hand blandar & chopper" as & is >>> > delimiter for solr query and thus this query is returning error. >>> > >>> > thanks in advance >>> > regards >>> > Rohan >>> >>> >> >> >
Re: spell suggestions help
Try replacing standard tokenizer with whitespace tokenizer in your field types. And make sure not to use any other token filters that might discard special characters (or provide a character map if they support one.) Also, be side to try your test terms in the Solr Admin UI ANalyzer page to see that the "&" is preserved or which stage in term analysis it gets discarded. -- Jack Krupansky -Original Message- From: Rohan Thakur Sent: Thursday, April 11, 2013 7:39 AM To: solr-user@lucene.apache.org Subject: Re: spell suggestions help urlencode replaces & with space thus resulting in results that contains even the single terms like in the case of mobile & accessories it replaces it with mobile accessories and results the document containing even accessories which i dont want. how to tackle this I tried using pattern replace filter at query time to replace & with and but it did not worked I used & => replace with "and" in this but did not worked any guess our help.. thanks regards rohan On Thu, Apr 11, 2013 at 4:39 PM, Rohan Thakur wrote: hi erick do we have to do urlencoding from the php side or does solr supports urlencode? On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson wrote: Try URL encoding it and/or escaping the & On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur wrote: > hi all > > one thing I wanted to clear is for every other query I have got correct > suggestions but these 2 cases I am not getting what suppose to be the > suggestions: > > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed > in direct solr spell cheker..but when I query for cattle I get cable as > only suggestion and not kettle why is this happening i want to get kettle > in suggestion as well im using jarowinkler distance according to which > score for cattle => cable which is coming out to be 0.857 and for cattle => > kettle which is coming out to be 0.777 kettle should also come in > suggestions but its not how can I correct this any one. > > 2) how to query for sentence like "hand blandar & chopper" as & is > delimiter for solr query and thus this query is returning error. > > thanks in advance > regards > Rohan
Re: spell suggestions help
urlencode replaces & with space thus resulting in results that contains even the single terms like in the case of mobile & accessories it replaces it with mobile accessories and results the document containing even accessories which i dont want. how to tackle this I tried using pattern replace filter at query time to replace & with and but it did not worked I used & => replace with "and" in this but did not worked any guess our help.. thanks regards rohan On Thu, Apr 11, 2013 at 4:39 PM, Rohan Thakur wrote: > hi erick > > do we have to do urlencoding from the php side or does solr supports > urlencode? > > > On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson > wrote: > >> Try URL encoding it and/or escaping the & >> >> On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur >> wrote: >> > hi all >> > >> > one thing I wanted to clear is for every other query I have got correct >> > suggestions but these 2 cases I am not getting what suppose to be the >> > suggestions: >> > >> > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word >> indexed >> > in direct solr spell cheker..but when I query for cattle I get cable as >> > only suggestion and not kettle why is this happening i want to get >> kettle >> > in suggestion as well im using jarowinkler distance according to which >> > score for cattle => cable which is coming out to be 0.857 and for >> cattle => >> > kettle which is coming out to be 0.777 kettle should also come in >> > suggestions but its not how can I correct this any one. >> > >> > 2) how to query for sentence like "hand blandar & chopper" as & is >> > delimiter for solr query and thus this query is returning error. >> > >> > thanks in advance >> > regards >> > Rohan >> > >
Re: spell suggestions help
hi erick do we have to do urlencoding from the php side or does solr supports urlencode? On Thu, Apr 11, 2013 at 5:57 AM, Erick Erickson wrote: > Try URL encoding it and/or escaping the & > > On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur wrote: > > hi all > > > > one thing I wanted to clear is for every other query I have got correct > > suggestions but these 2 cases I am not getting what suppose to be the > > suggestions: > > > > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed > > in direct solr spell cheker..but when I query for cattle I get cable as > > only suggestion and not kettle why is this happening i want to get kettle > > in suggestion as well im using jarowinkler distance according to which > > score for cattle => cable which is coming out to be 0.857 and for cattle > => > > kettle which is coming out to be 0.777 kettle should also come in > > suggestions but its not how can I correct this any one. > > > > 2) how to query for sentence like "hand blandar & chopper" as & is > > delimiter for solr query and thus this query is returning error. > > > > thanks in advance > > regards > > Rohan >
Re: spell suggestions help
Try URL encoding it and/or escaping the & On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur wrote: > hi all > > one thing I wanted to clear is for every other query I have got correct > suggestions but these 2 cases I am not getting what suppose to be the > suggestions: > > 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed > in direct solr spell cheker..but when I query for cattle I get cable as > only suggestion and not kettle why is this happening i want to get kettle > in suggestion as well im using jarowinkler distance according to which > score for cattle => cable which is coming out to be 0.857 and for cattle => > kettle which is coming out to be 0.777 kettle should also come in > suggestions but its not how can I correct this any one. > > 2) how to query for sentence like "hand blandar & chopper" as & is > delimiter for solr query and thus this query is returning error. > > thanks in advance > regards > Rohan
spell suggestions help
hi all one thing I wanted to clear is for every other query I have got correct suggestions but these 2 cases I am not getting what suppose to be the suggestions: 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed in direct solr spell cheker..but when I query for cattle I get cable as only suggestion and not kettle why is this happening i want to get kettle in suggestion as well im using jarowinkler distance according to which score for cattle => cable which is coming out to be 0.857 and for cattle => kettle which is coming out to be 0.777 kettle should also come in suggestions but its not how can I correct this any one. 2) how to query for sentence like "hand blandar & chopper" as & is delimiter for solr query and thus this query is returning error. thanks in advance regards Rohan
Re: solr spell suggestions help
hi all I have resolved all issues(its was relating to the distance measures I was using was by default lavanstine which is very basic and is not good now I am using jarowinkler distance measures which is better and now giving exact results that I was looking for) except the 4th one which I think is solrs issue and they have also released patch for that https://issues.apache.org/jira/browse/SOLR-2585 I am applying this patch now will let you know if its is working correctly. thanks regards Rohan On Fri, Apr 5, 2013 at 4:44 PM, Rohan Thakur wrote: > hi all > > I had some issues with solr spell suggestions. > > 1) first of all I wanted to know is indexbased spell suggestions better > then directspell suggestions that solr 4.1 provides in any way? > > 2) then I wanted to know is their way I can get suggestions for words > providing only few prefix for the word. like when I query sam I should get > samsung as one of suggestion. > > 3) also I wanted to know why am I not getting suggestions for the words > that have more then 2 character difference between them like if I query for > wirlpool wich has 8 characters I get suggestion as whirlpool which is 9 > characters and correct spelling but when I query for wirlpol which is 7 > characters it says that this is false spelling but does not show any > suggestions. even like if I search for pansonic(8 char) it provides > panasonic(9 char) as suggestion but when I remove one more character that > is is search for panonic(7 char) it does not return any suggestions?? how > can I correct this? even when I search for ipo it does not return ipod as > suggestions? > > 4) one more thing I want to get clear that when I search for microwave > ovan it does not give any miss spell even when ovan is wrong it provides > the result for microwave saying the query is correct...this is the case > when one of the term in the query is correct while others are incorrect it > does not point out the wrong spelling one but reutrns the result for > correct word thats it how can I correct this? similar is the case when I > query for microvave oven is shows the result for oven saying that the query > is correct.. > > 5) one more case is when I query plntronies (correct word is: plantronics) > it does not return any solution but when I query for plantronies it returns > the plantronics as suggestions why is that happening? > > *my schema.xml is:* > omitNorms="true"> > >pattern="\\\[\]\(\)\-\,\/\+" replacement=" "/> > > >words="stopwords.txt"/> > > > > > > >synonyms="synonyms.txt" ignoreCase="true" expand="true"/> >words="stopwords.txt"/> > > > > > > > > > > > *my solrconfig.xml is :* > > > > > > > > > > > *default* > > solr.DirectSolrSpellChecker > > > *spell > internal > > 0.3 > > 1 > > 1 > > 5 > > 4 > > 0.01 > > * > > > * > wordbreak > solr.WordBreakSolrSpellChecker > spell > true > true > 3 > > * > > > > * > jarowinkler > spell > solr.DirectSolrSpellChecker > name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance >* > > > > > > > > > > >*tSpell > * > > > > > * class="solr.SpellingQueryConverter"/>* > > > * startup="lazy"> > > spell > > default > wordbreak > > > > false > on > true > 10 > 5 > 5 > true > true > 10 > 5 > > > spellcheck > > > * > > > > thanks in advance > regards > Rohan >
solr spell suggestions help
hi all I had some issues with solr spell suggestions. 1) first of all I wanted to know is indexbased spell suggestions better then directspell suggestions that solr 4.1 provides in any way? 2) then I wanted to know is their way I can get suggestions for words providing only few prefix for the word. like when I query sam I should get samsung as one of suggestion. 3) also I wanted to know why am I not getting suggestions for the words that have more then 2 character difference between them like if I query for wirlpool wich has 8 characters I get suggestion as whirlpool which is 9 characters and correct spelling but when I query for wirlpol which is 7 characters it says that this is false spelling but does not show any suggestions. even like if I search for pansonic(8 char) it provides panasonic(9 char) as suggestion but when I remove one more character that is is search for panonic(7 char) it does not return any suggestions?? how can I correct this? even when I search for ipo it does not return ipod as suggestions? 4) one more thing I want to get clear that when I search for microwave ovan it does not give any miss spell even when ovan is wrong it provides the result for microwave saying the query is correct...this is the case when one of the term in the query is correct while others are incorrect it does not point out the wrong spelling one but reutrns the result for correct word thats it how can I correct this? similar is the case when I query for microvave oven is shows the result for oven saying that the query is correct.. 5) one more case is when I query plntronies (correct word is: plantronics) it does not return any solution but when I query for plantronies it returns the plantronics as suggestions why is that happening? *my schema.xml is:* *my solrconfig.xml is :* *default* solr.DirectSolrSpellChecker *spell internal 0.3 1 1 5 4 0.01 * * wordbreak solr.WordBreakSolrSpellChecker spell true true 3 * * jarowinkler spell solr.DirectSolrSpellChecker org.apache.lucene.search.spell.JaroWinklerDistance * *tSpell * * * * spell default wordbreak false on true 10 5 5 true true 10 5 spellcheck * thanks in advance regards Rohan
