Re: Multi-word Synonyms not working properly with Edismax
Yes, we tried that and it worked. We removed only for query analyzer and it is working properly now. On Wed, Sep 9, 2020 at 2:24 AM Dominique Bejean wrote: > Hi, > > Can you try to remove the RemoveDuplicatesTokenFilter ? > > Dominique > > Le mar. 8 sept. 2020 à 13:52, Manish Bafna a > écrit : > > > Hi, > > > > We are using the following configuration: > > > > > > > > -- > > > > *Schema: * > > > > > > > positionIncrementGap="100" autoGeneratePhraseQueries="true" > > > > omitNorms="true"> > > > > > > > > > > > > > > > > > > > > > > > > > > > dictionary="../hunspell_dictionary/en_US.dic" > > > > affix="../hunspell_dictionary/en_US.aff" ignoreCase="true" /> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > dictionary="../hunspell_dictionary/en_US.dic" > > > > affix="../hunspell_dictionary/en_US.aff" ignoreCase="true" /> > > > > > > > > > > > > > > > > > > > > *Managed Synonyms:* "abc implement", "bike", "xyz traders", "xyz > > transport" > > > > - > > > > *Query*: bike > > > > *parser Type:* edismax > > > > - > > > > *Parsed query (from debug)* : +DisjunctionMaxQueryfield1:"abc > > > > implement" field1:bike field1:"xyz traders" field1:"xyz trade")) > > > > - > > > > > > > > If you notice, there are 2 multi-word keywords starting with xyz, but > only > > > > 1 of them is getting added to the query. If we change xyz transport to xy > > > > transport, then it works properly. The issue is only when the 2 > multi-word > > > > keywords start with the same word. Though we are using graph synonyms, it > > > > is not working properly. > > > > > > > > Are we doing anything wrong here? > > > > > > > > Thanks, > > > > Manish. > > > > >
Re: Multi-word Synonyms not working properly with Edismax
Hi, Can you try to remove the RemoveDuplicatesTokenFilter ? Dominique Le mar. 8 sept. 2020 à 13:52, Manish Bafna a écrit : > Hi, > > We are using the following configuration: > > > > -- > > *Schema: * > > > positionIncrementGap="100" autoGeneratePhraseQueries="true" > > omitNorms="true"> > > > > > > > > > > > > > dictionary="../hunspell_dictionary/en_US.dic" > > affix="../hunspell_dictionary/en_US.aff" ignoreCase="true" /> > > > > > > > > > > > > > > > > > > dictionary="../hunspell_dictionary/en_US.dic" > > affix="../hunspell_dictionary/en_US.aff" ignoreCase="true" /> > > > > > > > > > > *Managed Synonyms:* "abc implement", "bike", "xyz traders", "xyz > transport" > > - > > *Query*: bike > > *parser Type:* edismax > > - > > *Parsed query (from debug)* : +DisjunctionMaxQueryfield1:"abc > > implement" field1:bike field1:"xyz traders" field1:"xyz trade")) > > - > > > > If you notice, there are 2 multi-word keywords starting with xyz, but only > > 1 of them is getting added to the query. If we change xyz transport to xy > > transport, then it works properly. The issue is only when the 2 multi-word > > keywords start with the same word. Though we are using graph synonyms, it > > is not working properly. > > > > Are we doing anything wrong here? > > > > Thanks, > > Manish. > >
Multi-word Synonyms not working properly with Edismax
Hi, We are using the following configuration: -- *Schema: * *Managed Synonyms:* "abc implement", "bike", "xyz traders", "xyz transport" - *Query*: bike *parser Type:* edismax - *Parsed query (from debug)* : +DisjunctionMaxQueryfield1:"abc implement" field1:bike field1:"xyz traders" field1:"xyz trade")) - If you notice, there are 2 multi-word keywords starting with xyz, but only 1 of them is getting added to the query. If we change xyz transport to xy transport, then it works properly. The issue is only when the 2 multi-word keywords start with the same word. Though we are using graph synonyms, it is not working properly. Are we doing anything wrong here? Thanks, Manish.
Re: synonyms not working
Attach debug=query to your URL and inspect the parsed query, you should be seeing the substitutions if you're configured correctly. Multi-word synonyms at query time have the getting through the query parser problem. Best Erick On Wed, Sep 11, 2013 at 11:04 AM, cheops m.schm...@mediaskill.de wrote: Hi, I'm using solr4.4 and try to use different synonyms based on different fieldtypes: fieldType name=text_general class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.LowerCaseFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType ...I have the same fieldtype for english (name=text_general_en and synonyms=synonyms_en.txt). The first fieldtype works fine, my synonyms are processed and the result is as expected. But the en-version doesn't seem to work. I'm able to find the original english words but the synonyms are not processed. ps: yes, i know using synonyms at query time is not a good idea :-) ... but can't change it here Any help would be appreciated! Thank you. Best regards Marcus -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-tp4089318.html Sent from the Solr - User mailing list archive at Nabble.com.
synonyms not working
Hi, I'm using solr4.4 and try to use different synonyms based on different fieldtypes: fieldType name=text_general class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.LowerCaseFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.StandardTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType ...I have the same fieldtype for english (name=text_general_en and synonyms=synonyms_en.txt). The first fieldtype works fine, my synonyms are processed and the result is as expected. But the en-version doesn't seem to work. I'm able to find the original english words but the synonyms are not processed. ps: yes, i know using synonyms at query time is not a good idea :-) ... but can't change it here Any help would be appreciated! Thank you. Best regards Marcus -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-tp4089318.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: synonyms not working
thanx for your help. could solve the problem meanwhile! i used analyzer type=query_en ...which is wrong, it must be analyzer type=query -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-tp4089318p4089345.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Synonyms Not Working when using SRC DEST
I have a very huge schema spanning up to 10K lines , if I use query time it will be huge hit for me because one term will be mapped to multiple terms . similar in the case of allergy I think maybe you mean synonym file, rather than the schema? I doubt that the number of lines matters all that much, though undoubtedly some. I expect that Solr loads that synonym file into some kind of hash map, rather than searching it linearly -- though I have not looked at the code for that. I replace allergy during the index with doctors , So it shouldn't be part of the document ? Yes indeed, doctors would be in the index, and would give you a hit on that document when searched. But because your synonym file specifies replacement, that means that allergy is *NOT* part of the index, hence, when you searched on allergy, you got no results. As far as synonym expansion being a huge hit, no, not really, I think. Besides, if you are not getting what you want or need, speed becomes pretty much irrelevant. We did some performance testing: modest single server (i.e., a laptop running Windows XP with only 2GB total memory available), pretty much configured out of the box with jetty, except that we added waffle authentication. The data was names, addresses and the like (not text) -- 7+ million rows, with considerable synonym expansion: 200 first name synonyms, 433 last name synonyms, expanded at both index time and search time. We then did a search test driven from those same synonyms files, by randomly picking out a name from the first and last name list, the idea being that most likely names did have some synonyms. Under Solr 3.1, once the OS file system cache got some entries in there, running with 8 concurrent client search threads sending HTTP search requests (done in perl) we averaged about .50 seconds per request, or over 55,000 searches per hour. JRJ -Original Message- From: balaji [mailto:mcabal...@gmail.com] Sent: Tuesday, September 06, 2011 7:48 PM To: solr-user@lucene.apache.org Subject: Re: Synonyms Not Working when using SRC DEST It won't work given your current schema. To get the desired results, you would need to expand your synonyms at both index AND query time. Right now your schema seems to specify it only at index time. I have a very huge schema spanning up to 10K lines , if I use query time it will be huge hit for me because one term will be mapped to multiple terms . similar in the case of allergy I doesn't want to go with comma separated as it will give some erroneous results and more over allergy and doctors are not equivalent terms to be used in comma So, as the other respondent indicated, currently you replace allergy with the other list when indexing, and since allergy is not replaced during query, it gets no hits. I replace allergy during the index with doctors , So it shouldn't be part of the document ? Thanks Balaji -- View this message in context: http://lucene.472066.n3.nabble.com/Synonyms-Not-Working-when-using-SRC-DEST-tp3313862p3315287.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Synonyms Not Working when using SRC DEST
Also, just to make one thing just a bit more clear. You can specify two different kinds of entries in synonym files. See http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters (solr.SynonymFilterFactory) One is replacement, where the words before the = are *replaced* by the right hand side, i.e., the words on the left hand side disappear. This is what you are currently doing according to your original message: #Explicit mappings match any token sequence on the LHS of = #and replace with all alternatives on the RHS. These types of mappings #ignore the expand parameter in the schema. #Examples: i-pod, i pod = ipod, sea biscuit, sea biscit = seabiscuit The other is equivalence, where each term is expanded into the entire list, if you do the following, with expand set to true: #Equivalent synonyms may be separated with commas and give #no explicit mapping. In this case the mapping behavior will #be taken from the expand parameter in the schema. This allows #the same synonym file to be used in different synonym handling strategies. #Examples: ipod, i-pod, i pod foozball , foosball universe , cosmos So, if instead of: allergy test = Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons You specified allergy test = allergy test, Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons Or allergy test, Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons with expand set to true, then you might get the behavior your desire: Allergy test would get indexed, along with Doctors and all of the rest. The difference being that in the second case, any of those terms (e.g. Docotrs) would also get indexed as Allergy test which might not be what you desire, in which case the first one would do what you want. I expect that all you really need to do is: allergy test = allergy test, Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons to solve your problem. JRJ -Original Message- From: balaji [mailto:mcabal...@gmail.com] Sent: Tuesday, September 06, 2011 7:48 PM To: solr-user@lucene.apache.org Subject: Re: Synonyms Not Working when using SRC DEST It won't work given your current schema. To get the desired results, you would need to expand your synonyms at both index AND query time. Right now your schema seems to specify it only at index time. I have a very huge schema spanning up to 10K lines , if I use query time it will be huge hit for me because one term will be mapped to multiple terms . similar in the case of allergy I doesn't want to go with comma separated as it will give some erroneous results and more over allergy and doctors are not equivalent terms to be used in comma So, as the other respondent indicated, currently you replace allergy with the other list when indexing, and since allergy is not replaced during query, it gets no hits. I replace allergy during the index with doctors , So it shouldn't be part of the document ? Thanks Balaji -- View this message in context: http://lucene.472066.n3.nabble.com/Synonyms-Not-Working-when-using-SRC-DEST-tp3313862p3315287.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Synonyms Not Working when using SRC DEST
So, if instead of: allergy test = Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons You specified allergy test = allergy test, Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons I followed the above approach allergy test = allergy test, Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons and it works as expected , Thanks for making it more clear Thanks Balaji -- View this message in context: http://lucene.472066.n3.nabble.com/Synonyms-Not-Working-when-using-SRC-DEST-tp3313862p3316691.html Sent from the Solr - User mailing list archive at Nabble.com.
Synonyms Not Working when using SRC DEST
Hi all The Question might sound stupid . I have a large synonym file and have created the synonyms something like below *allergy test = Doctors, Doctors-Medical, PHYSICIANS, Physicians Surgeons * I have also added the synonym to get indexed during index time like below fieldType name=text class=solr.TextField positionIncrementGap=100 analyzer type=index charFilter class=solr.MappingCharFilterFactory mapping=mapping-ISOLatin1Accent.txt/ tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.SnowballPorterFilterFactory language=English protected=protwords.txt/ filter class=solr.RemoveDuplicatesTokenFilterFactory/ /analyzer analyzer type=query charFilter class=solr.MappingCharFilterFactory mapping=mapping-ISOLatin1Accent.txt/ tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=0 catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.SnowballPorterFilterFactory language=English protected=protwords.txt/ filter class=solr.RemoveDuplicatesTokenFilterFactory/ /analyzer /fieldType But when I do a search for allergy , I get 0 results when I go to solr/admin/analysis.jsp and I could see the corresponding synonym show up http://lucene.472066.n3.nabble.com/file/n3313862/Screenshot-1.png So I couldn't find where is the problem when i change the synonym file to a comma separated I am able to see the results It would be great if you can help me out with this Thanks Balaji -- View this message in context: http://lucene.472066.n3.nabble.com/Synonyms-Not-Working-when-using-SRC-DEST-tp3313862p3313862.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Synonyms Not Working when using SRC DEST
: *allergy test = Doctors, Doctors-Medical, PHYSICIANS, Physicians : Surgeons .. : analyzer type=index ... : filter class=solr.SynonymFilterFactory synonyms=synonyms.txt : ignoreCase=true expand=true/ ... : But when I do a search for allergy , I get 0 results You've configured your field so that any time the terms allergy and test appear in sequence in a field value you index, those terms are removed and replaced by new terms (Doctors, Doctors-Medical, etc...) So if the term allergy only appears in the source text followed by the term test then it will never actually be indexed in your document, so a serach for it will never match. You can see this exact behavior in the screen shot you posted of the analysis tool... : http://lucene.472066.n3.nabble.com/file/n3313862/Screenshot-1.png ...after the synonyn filter, the term allergy is not in your indexed terms. : when i change the synonym file to a comma separated I am able to see the : results because when using a comma instead of = you are saying if any of these term sequences exist, expand it to *all* of these term sequences. Please note the docs on SYnonymFilter, particularly the examples... https://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory -Hoss
Re: Synonyms Not Working when using SRC DEST
Hi Chris The Terms Doctors , Doctors-Medical are all present in my Document body, title fields etc.. but Allergy Test is not . So what I am doing in synonym file is if a user searches for allergy test bring me results that match Doctors etc.. i.e Explicit mappings match any token sequence on the LHS of = and replace with all alternatives on the RHS. So when I do a search allergy test it should map with doctors and should bring me results but it is not mapping . Is there any way I make it work Hope it clarifies Thanks Balaji -- View this message in context: http://lucene.472066.n3.nabble.com/Synonyms-Not-Working-when-using-SRC-DEST-tp3313862p3314222.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Synonyms Not Working when using SRC DEST
It won't work given your current schema. To get the desired results, you would need to expand your synonyms at both index AND query time. Right now your schema seems to specify it only at index time. So, as the other respondent indicated, currently you replace allergy with the other list when indexing, and since allergy is not replaced during query, it gets no hits. It almost sounds like a case where you could consider synonym expansion only at query time, rather than at index time (though that is usually not advisable for reasons discussed on the Wiki). Then Allergy would get expanded during a search, and hit the documents with Doctors, etc. JRJ -Original Message- From: balaji [mailto:mcabal...@gmail.com] Sent: Tuesday, September 06, 2011 12:24 PM To: solr-user@lucene.apache.org Subject: Re: Synonyms Not Working when using SRC DEST Hi Chris The Terms Doctors , Doctors-Medical are all present in my Document body, title fields etc.. but Allergy Test is not . So what I am doing in synonym file is if a user searches for allergy test bring me results that match Doctors etc.. i.e Explicit mappings match any token sequence on the LHS of = and replace with all alternatives on the RHS. So when I do a search allergy test it should map with doctors and should bring me results but it is not mapping . Is there any way I make it work Hope it clarifies Thanks Balaji -- View this message in context: http://lucene.472066.n3.nabble.com/Synonyms-Not-Working-when-using-SRC-DEST-tp3313862p3314222.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Synonyms Not Working when using SRC DEST
It won't work given your current schema. To get the desired results, you would need to expand your synonyms at both index AND query time. Right now your schema seems to specify it only at index time. I have a very huge schema spanning up to 10K lines , if I use query time it will be huge hit for me because one term will be mapped to multiple terms . similar in the case of allergy I doesn't want to go with comma separated as it will give some erroneous results and more over allergy and doctors are not equivalent terms to be used in comma So, as the other respondent indicated, currently you replace allergy with the other list when indexing, and since allergy is not replaced during query, it gets no hits. I replace allergy during the index with doctors , So it shouldn't be part of the document ? Thanks Balaji -- View this message in context: http://lucene.472066.n3.nabble.com/Synonyms-Not-Working-when-using-SRC-DEST-tp3313862p3315287.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: synonyms not working with copyfield
: fields during indexing. However, my search interface is just a text : box like Google and I need to take the query and return only those : documents that match ALL terms in the query and if I am going to take as mentioned previously in this thread: this is exactly what the dismax QParser was designed for. -Hoss
Re: synonyms not working with copyfield
Hi Surajit I aint sure if this is any help, but I had a similar problem but with stop words, they were not working with dismax queries. Well to cut a long story it seems that all the querying fields need to be configured with stopwords. Maybe this has the similar affect with Synonyms confguration, thus your copyField should be defined as a type that is configured with the SynonymFilterFactory, just like person_name. You can find some guidance here: http://bibwild.wordpress.com/2010/04/14/solr-stop-wordsdismax-gotcha/ Gary
Re: synonyms not working with copyfield
I have indexed person names in solr using synonym expansion and am getting a match when I explicitly use that field in my query (name:query). However, when I copy that field into another field using copyfield and search on that field, I don't get a match. Below are excerpts from schema.txt. I am new to Solr and appreciate any help! Thanks. Surajit fieldType name=person_name class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=0 catenateWords=1 catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.SynonymFilterFactory synonyms=person-synonyms.txt ignoreCase=true expand=true/ /analyzer /fieldType field name=sngr type=person_name multiValued=true indexed=true stored=true required=false / field name=text type=text indexed=true stored=true multiValued=true/ copyField source=sngr dest=text/ CopyField just copies raw text, i mean not analyzed. Do you have a filter class=solr.SynonymFilterFactory synonyms=person-synonyms.txt ignoreCase=true expand=true/ in your text fieldType definition?
Re: synonyms not working with copyfield
Thanks much! I added a synonym filter to the copyfield and it started working which is good, but the different fields that I copy into the copyfield need different analysis and I no longer am able to do that. I can, of course, search against the individual fields instead of the copyfield, but I want to return a match only if ALL terms in the query are matched in the overall document (as in an AND) and if I search against individual fields I am not sure of an easy way to figure out if all terms matched in the overall document. Any ideas? surajit -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-with-copyfield-tp814108p815263.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: synonyms not working with copyfield
take a look at the DismaxRequestHandler: http://wiki.apache.org/solr/DisMaxRequestHandler -Original Message- From: surajit surajit.em...@gmail.com To: solr-user@lucene.apache.org Sent: Thu, May 13, 2010 9:52 pm Subject: Re: synonyms not working with copyfield Thanks much! I added a synonym filter to the copyfield and it started working which is good, but the different fields that I copy into the copyfield need different analysis and I no longer am able to do that. I can, of course, search against the individual fields instead of the copyfield, but I want to return a match only if ALL terms in the query are matched in the overall document (as in an AND) and if I search against individual fields I am not sure of an easy way to figure out if all terms matched in the overall document. Any ideas? surajit -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-with-copyfield-tp814108p815263.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: synonyms not working with copyfield
: which is good, but the different fields that I copy into the copyfield need : different analysis and I no longer am able to do that. I can, of course, Fundementally, Solr can only apply a single analysis chain to all of the text in a given field -- regardless of where it may be copied from. if it didn't, there would be no way to get matches at query time. the query analysis has to make sense for the index analysis, so it has to be consistent. -Hoss
Re: synonyms not working with copyfield
Understood and I can work with that limitation by using separate fields during indexing. However, my search interface is just a text box like Google and I need to take the query and return only those documents that match ALL terms in the query and if I am going to take the query and match it against each field (separately), how do I get back documents matching all user terms? One soln I can think of is to take all the field-specific analysis out of solr and do it as a pre-process step, but want to make sure there isn't an alternative within Solr. surajit On Thu, May 13, 2010 at 12:42 PM, Chris Hostetter-3 [via Lucene] ml-node+815302-427668360-263...@n3.nabble.com wrote: : which is good, but the different fields that I copy into the copyfield need : different analysis and I no longer am able to do that. I can, of course, Fundementally, Solr can only apply a single analysis chain to all of the text in a given field -- regardless of where it may be copied from. if it didn't, there would be no way to get matches at query time. the query analysis has to make sense for the index analysis, so it has to be consistent. -Hoss View message @ http://lucene.472066.n3.nabble.com/synonyms-not-working-with-copyfield-tp814108p815302.html To unsubscribe from Re: synonyms not working with copyfield, click here. -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-with-copyfield-tp814108p815426.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: synonyms not working with copyfield
Hi, You could use a copyField against all fields and then AND the query terms given. Quite restrictive but all terms would then have to be present to match. I'm still a relative newbie to Solr so perhaps I'm horribly wrong. Cheers Nick On 13 May 2010, at 18:18, surajit wrote: Understood and I can work with that limitation by using separate fields during indexing. However, my search interface is just a text box like Google and I need to take the query and return only those documents that match ALL terms in the query and if I am going to take the query and match it against each field (separately), how do I get back documents matching all user terms? One soln I can think of is to take all the field-specific analysis out of solr and do it as a pre-process step, but want to make sure there isn't an alternative within Solr. surajit On Thu, May 13, 2010 at 12:42 PM, Chris Hostetter-3 [via Lucene] ml-node+815302-427668360-263...@n3.nabble.com wrote: : which is good, but the different fields that I copy into the copyfield need : different analysis and I no longer am able to do that. I can, of course, Fundementally, Solr can only apply a single analysis chain to all of the text in a given field -- regardless of where it may be copied from. if it didn't, there would be no way to get matches at query time. the query analysis has to make sense for the index analysis, so it has to be consistent. -Hoss View message @ http://lucene.472066.n3.nabble.com/synonyms-not-working-with-copyfield-tp814108p815302.html To unsubscribe from Re: synonyms not working with copyfield, click here. -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-with-copyfield-tp814108p815426.html Sent from the Solr - User mailing list archive at Nabble.com.
synonyms not working with copyfield
I have indexed person names in solr using synonym expansion and am getting a match when I explicitly use that field in my query (name:query). However, when I copy that field into another field using copyfield and search on that field, I don't get a match. Below are excerpts from schema.txt. I am new to Solr and appreciate any help! Thanks. Surajit fieldType name=person_name class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=0 catenateWords=1 catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.SynonymFilterFactory synonyms=person-synonyms.txt ignoreCase=true expand=true/ /analyzer /fieldType field name=sngr type=person_name multiValued=true indexed=true stored=true required=false / field name=text type=text indexed=true stored=true multiValued=true/ copyField source=sngr dest=text/ -- View this message in context: http://lucene.472066.n3.nabble.com/synonyms-not-working-with-copyfield-tp814108p814108.html Sent from the Solr - User mailing list archive at Nabble.com.
Phrase synonyms not working | Synonyms not in search results
We are facing following issues with our Solr 1.3.0 : 1. Phrase synonym not working – Search for multi word terms (phrase) is not working for our Solr configuration. Be it a one-way synonym or two-way, it’s not working for a phrase. For phrase search, we are enclosing the term as well as the synonyms in double quotes i.e. the phrase ‘cute cat’ is entered as “cute cat” with quotes in synonyms.txt file. The synonyms are also enclosed in quotes like “little cat” with quotes. When we perform a search for “cute cat” with quotes, we get the results for ‘cute cat’ but not for ‘little cat even if it is defined as a synonym. However, if we define a single word as synonym for a set of synonyms, we get all the expected results i.e. if we define a set of synonyms as ‘”cute cat”, “little cat”, ”felines”’, a search for “felines” with quotes or without quotes will provide us a result which will include the results for “little cat”, “cute cat” and “feline” but a search on “little cat” or “cute cat” will return the result containing the search results for the precise term used for search. When searching for a phrase without quotes, we get all the results for the individual words and combination of words, just the regular search result. We need the search results for the precise combination of words in the phrase along with the synonyms for the phrase. The “expand” parameter is currently set to “true” but setting it to false also did not help with phrase synonym search. We are using the DisjunctionMaxQueryParser. 2. Search results not correct when a term is searched as a phrase – As per our requirements, when a term is searched as a phrase, i.e. enclosed in double quotes, it should return results pertaining to that specific word. The results should not contain the search results for the synonyms of the search term. However, we are getting the results for the terms defined as the synonyms also. Has anybody got some experience with similar kind of problems? For our first problem, we are looking for some configuration settings in schema.xml which may help us getting the phrase synonyms work. -- View this message in context: http://www.nabble.com/Phrase-synonyms-not-working-%7C-Synonyms-not-in-search-results-tp25104201p25104201.html Sent from the Solr - User mailing list archive at Nabble.com.
