Re: How to configure solr while having Apostrophes in fields
Thanks Eric , I have put analyser screen shot ,and i am not sure what is the problem in WCF. <http://lucene.472066.n3.nabble.com/file/n4289329/solr-analyser.png> -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-configure-solr-while-having-Apostrophes-in-fields-tp4289196p4289329.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to configure solr while having Apostrophes in fields
I'd _strongly_ recommend you become familiar with the
admin>>(your core)>>analysis page. It tells you exactly
what each filter does to your input and makes it much
simpler to answer questions like this. Hover over each
of the gray letter pairs (e.g. "SF" will be gray, hover over
it and you'll see that that's the "StopFilter").
In this case WordDelimiterFilterFactory is breaking
on on all non alpha-numerics. Do note that when
you remove it, all the _other_ punctuation that it
strips will suddenly be relevant, i.e.
"my dog has fleas.", the period after "fleas" will
be part of that token so you'll have to deal with that.
Best,
Erick
On Wed, Jul 27, 2016 at 4:58 AM, nitin.garg88 <nitin.gar...@gmail.com> wrote:
> When i search for "plato" it return me all records with
> "plato,platos,plato's"
> When i search for "platos" it return me all records with "platos,plato's"
> When i search for "plato's" it return me all records with "platos,plato's"
>
> Please suggest me how to configure schema.xml .Below is my "text" setting in
> schema.xml
>
>autoGeneratePhraseQueries="true">
>
>
>
>
>
>
>
> ignoreCase="true"
>
> synonyms="${home}/solr-configuration/bibliographic-protected-synonyms.txt"/>
>
>
> generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="1" catenateAll="0"
> stemEnglishPossessive="0" splitOnCaseChange="0"
>
> protected="${home}/solr-configuration/bibliographic-protwords.txt"/>
>
>
> enablePositionIncrements="true"
>
> words="${home}/solr-configuration/bibliographic-stopwords.txt"/>
>
>
> ignoreCase="true"
>
> synonyms="${home}/solr-configuration/bibliographic-synonyms.txt"/>
>
>
> mappingFile="${home}/sort.map"/>
>
>
>
>
> ignoreCase="true"
>
> synonyms="${home}/solr-configuration/bibliographic-protected-synonyms.txt"/>
> generateWordParts="1" generateNumberParts="1"
> catenateWords="0" catenateNumbers="0" catenateAll="0"
> stemEnglishPossessive="0" splitOnCaseChange="0"
>
> protected="${home}/solr-configuration/bibliographic-protwords.txt" />
> enablePositionIncrements="true"
>
> words="${home}/solr-configuration/bibliographic-stopwords.txt"/>
> ignoreCase="true"
>
> synonyms="${home}/solr-configuration/bibliographic-synonyms.txt"/>
> mappingFile="${home}/sort.map"/>
>
>
>
> Thanks in advance !
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/How-to-configure-solr-while-having-Apostrophes-in-fields-tp4289196.html
> Sent from the Solr - User mailing list archive at Nabble.com.
How to configure solr while having Apostrophes in fields
When i search for "plato" it return me all records with "plato,platos,plato's" When i search for "platos" it return me all records with "platos,plato's" When i search for "plato's" it return me all records with "platos,plato's" Please suggest me how to configure schema.xml .Below is my "text" setting in schema.xml Thanks in advance ! -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-configure-solr-while-having-Apostrophes-in-fields-tp4289196.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Apostrophes in fields
in my case - the fields with apostrophe not returned in results When I search for -- dev it shows me following results dev dev's devendra but when I search for -- dev' (dev with apo only) Nothing comes out as result ? What could be the workaround ? Thanks Devendra -- View this message in context: http://lucene.472066.n3.nabble.com/Apostrophes-in-fields-tp475058p4087910.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Apostrophes in fields
On 9/3/2013 3:59 AM, devendra W wrote: in my case - the fields with apostrophe not returned in results Don't use special characters in field names. If it wouldn't work as an variable name, function name (or other identifier) in a typical programming language (Java, C, Perl), then it will probably cause you problems with a field name. This basically means: 7-bit ASCII only. Starts with a letter, contains only letters, numbers, and the underscore. Most punctuation other than the underscore has a special meaning to Solr. Using extended characters (UTF-8, or those beyond 7-bit ASCII) *might* work, but it's fairly easy to screw that up and use the wrong character set, so it's better if you just don't do it. Thanks, Shawn
Re: Apostrophes in fields
Show us your full field type with analyzer. I suspect that the problem is that one of the index-time filters is turning dev's into devs (WDF does that), but at query-time there is no filter that removes a trailing apostrophe. Use the Solr Admin UI Analysis page to see home dev's gets indexed and how dev' gets analyzed at query time. -- Jack Krupansky -Original Message- From: devendra W Sent: Tuesday, September 03, 2013 5:59 AM To: solr-user@lucene.apache.org Subject: Re: Apostrophes in fields in my case - the fields with apostrophe not returned in results When I search for -- dev it shows me following results dev dev's devendra but when I search for -- dev' (dev with apo only) Nothing comes out as result ? What could be the workaround ? Thanks Devendra -- View this message in context: http://lucene.472066.n3.nabble.com/Apostrophes-in-fields-tp475058p4087910.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Apostrophes in fields
: This problem is why some sloppiness is recommended when dealing with : WordDelimiterFilter. particularly when using the generate___Parts=true options Nick: if you want simpler matching like this, you might want to consider simplifying your definition of text ... if you look at the textTight fieldtype in the example shema (used by the field sku) you'll see a simpler usage of WordDelimiterFilter ... alternately you may just want to use lucene's basic StandardAnalzyer ... i believe it strips Apostrophes. as a real last resort, you could use the recently added PatternReplaceFilter to strip out apostrophe's prior to WordDelimiterFilter (if you like everything WordDelim does for you except spliting on apostrophes) : - optionally index ohara at *both* o and hara then searching for Shelley ohara memorial fails without unless yo have slop .. if you need slop, you might as well not index it twice (not to mention it throws off the tf/idf calculations) : - pick the alignment based on the token position in the stream... : right-justify the catenations if it's the first token, otherwise : left-justify. One could try to identify proper names and do the : justification correctly too (blech). oh for the love of god please no. -Hoss
Re: Apostrophes in fields
Using the fuzzy searching fixed the problem - I will have a play with the analzyers and see if I can get it working nicely. Thanks again, much apreciated. On 1/17/07, Chris Hostetter [EMAIL PROTECTED] wrote: : This problem is why some sloppiness is recommended when dealing with : WordDelimiterFilter. particularly when using the generate___Parts=true options Nick: if you want simpler matching like this, you might want to consider simplifying your definition of text ... if you look at the textTight fieldtype in the example shema (used by the field sku) you'll see a simpler usage of WordDelimiterFilter ... alternately you may just want to use lucene's basic StandardAnalzyer ... i believe it strips Apostrophes. as a real last resort, you could use the recently added PatternReplaceFilter to strip out apostrophe's prior to WordDelimiterFilter (if you like everything WordDelim does for you except spliting on apostrophes) : - optionally index ohara at *both* o and hara then searching for Shelley ohara memorial fails without unless yo have slop .. if you need slop, you might as well not index it twice (not to mention it throws off the tf/idf calculations) : - pick the alignment based on the token position in the stream... : right-justify the catenations if it's the first token, otherwise : left-justify. One could try to identify proper names and do the : justification correctly too (blech). oh for the love of god please no. -Hoss -- - Nick
Apostrophes in fields
Hi This is probably more of a lucene question, but: I have an author field, If I query author:Shelley Ohara - no results are returned If I query author:Shelley O'hara - many results are returned, Is it possible, to get solr to ignore apostrophes in queries like the one above? e.g. doc doc arr name=authorstrShelley O'Hara/str/arr bool name=availabletrue/bool str name=descriptionlong descirption/str str name=ean9780764559747/str str name=formatPaperback/str str name=publisherIDGP/str str name=titleKierkegaard Within Your Grasp/str str name=year2004/str /doc Thanks -- - Nick
