This has come up with stemming: you can stem your synonym list with the FieldAnalyzer Solr http call, then save the final chewed-up terms as a new synonym file. You then use that one in the analyzer stack below the stemmer filter.
On Mon, Apr 25, 2011 at 9:15 PM, Otis Gospodnetic <otis_gospodne...@yahoo.com> wrote: > Hi Otis & Robert, > > ----- Original Message ---- > >> >> How do people handle cases where synonyms are used and there are multiple >> version of the original word that really need to point to the same set of >> synonyms? >> >> For example: >> Consider singular and plural of the word "responsibility". One might have >> synonyms defined like this: >> >> responsibility, obligation, duty >> >> But the plural "responsibilities" is not in there, and thus it will not get >> expanded to the synonyms above! That's a problem. >> >> Sure, one could change the synonyms file to look like this: >> >> responsibility, responsibilities, obligation, duty >> >> But that means somebody needs to think of all variations of the word! > > Yes, that seems to be the case now, as it was in 2008: > http://search-lucene.com/m/gLwUCV0qU02&subj=Re+Synonyms+and+stemming+revisited > http://search-lucene.com/m/7lqdp1ldrqx (Hoss replied, but I think that > suggestion doesn't actually work) > >> Is there a something one can do to get all variations of the word to map to >>the >> >> same synonyms without having to explicitly specify all variations of the > word? > > I think this is where Robert's 2+2lemma pointer may help because the 2+lemma > list contains "records" where a headword is followed by a list of other > variations of the word. The way I think this would help is by simply taking > that list and turning it into the synonyms file format, and then merging in > the > actual synonyms. > > For example, if I have the word "responsibility", then from 2+2lemma I should > be > able to get that "responsibilities" is one of the variants of > "responsibility". > I should then be able to take those 2 words and stick them in synonyms file > like > this: > > responsibility, responsibilities > > And then append actual synonyms to that: > > responsibility, responsibilities, obligation, duty > > But I may then need to actually expand synonyms themselves, too (again using > data from 2+2lemma): > > responsibility, responsibilities, obligation, obligations, duty, duties > > > I haven't tried this yet. Just theorizing and hoping for feedback. > > Does this sound about right? > > Thanks, > Otis > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > -- Lance Norskog goks...@gmail.com
