Hmm, I've gotten this very wrong :) - DisjunctionMaxQuery will operate per-doc, so using it in the way I suggested will not allow for synonym IDF leveling across documents. Also, scoring obviously includes more factors than IDF.
On Dec 12, 2012, at 5:18 PM, Steve Rowe <sar...@gmail.com> wrote: > But couldn't the IDF problem be fixed by applying the same IDF to all > synonyms, e.g. via DisjunctionMaxQuery? (Maybe the ideal would be an > average, not a max.) > > (E)dismax applies this query per-field, but AFAICT there is nothing stopping > anybody (modulo query parser construction :) ) from using it on synonyms in > the same field. > > Steve > > On Dec 12, 2012, at 12:50 PM, Walter Underwood <wun...@wunderwood.org> wrote: > >> Query parsers cannot fix the IDF problem or make query-time synonyms faster. >> Query synonym expansion makes more search terms. More search terms are more >> work at query time. >> >> The IDF problem is real; I've run up against it. The most rare variant of >> the synonym have the highest score. This probably the opposite of what you >> want. For me, it was "TV" and "television". Documents with "TV" had higher >> scores than those with "television". >> >> wunder >> >> On Dec 12, 2012, at 9:45 AM, Roman Chyla wrote: >> >>> @wunder >>> It is a misconception (well, supported by that wiki description) that the >>> query time synonym filter have these problems. It is actually the default >>> parser, that is causing these problems. Look at this if you still think >>> that index time synonyms are cure for all: >>> https://issues.apache.org/jira/browse/LUCENE-4499 >>> >>> @joe >>> If you can use the flexible query parser (as linked in by @Swati) then all >>> you need to do is to define a different field with a different tokenizer >>> chain and then swap the field names before the analyzers processes the >>> document (and then rewrite the field name back - for example, we have >>> fields called "author" and "author_nosyn") >>> >>> roman >>> >>> On Wed, Dec 12, 2012 at 12:38 PM, Walter Underwood >>> <wun...@wunderwood.org>wrote: >>> >>>> Query time synonyms have known problems. They are slower, cause incorrect >>>> IDF, and don't work for phrase synonyms. >>>> >>>> Apply synonyms at index time and you will have none of those problems. >>>> >>>> See: >>>> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory >>>> >>>> wunder >>>> >>>> On Dec 12, 2012, at 9:34 AM, Swati Swoboda wrote: >>>> >>>>> Query-time analyzers are still applied, even if you include a string in >>>> quotes. Would you expect "foo" to not match "Foo" just because it's >>>> enclosed in quotes? >>>>> >>>>> Also look at this, someone who had similar requirements: >>>>> >>>> http://lucene.472066.n3.nabble.com/Synonym-Filter-disable-at-query-time-td2919876.html >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: joe.cohe...@gmail.com [mailto:joe.cohe...@gmail.com] >>>>> Sent: Wednesday, December 12, 2012 12:09 PM >>>>> To: solr-user@lucene.apache.org >>>>> Subject: Re: Can a field with defined synonym be searched without the >>>> synonym? >>>>> >>>>> >>>>> I'm aplying only query-time synonym, so I have the original values >>>> stored and indexed. >>>>> I would've expected that if I search a strin with quotations, i'll get >>>> the exact match, without applying a synonym. >>>>> >>>>> any way to achieve that? >>>>> >>>>> >>>>> Upayavira wrote >>>>>> You can only search against terms that are stored in your index. If >>>>>> you have applied index time synonyms, you can't remove them at query >>>> time. >>>>>> >>>>>> You can, however, use copyField to clone an incoming field to another >>>>>> field that doesn't use synonyms, and search against that field instead. >>>>>> >>>>>> Upayavira >>>>>> >>>>>> On Wed, Dec 12, 2012, at 04:26 PM, >>>>> >>>>>> joe.cohen.m@ >>>>> >>>>>> wrote: >>>>>>> Hi >>>>>>> I hava a field type without defined synonym.txt which retrieves both >>>>>>> records with "home" and "house" when I search either one of them. >>>>>>> >>>>>>> I want to be able to search this field on the specific value that I >>>>>>> enter, without the synonym filter. >>>>>>> >>>>>>> is it possible? >>>>>>> >>>>>>> thanks. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> View this message in context: >>>>>>> http://lucene.472066.n3.nabble.com/Can-a-field-with-defined-synonym-b >>>>>>> e-searched-without-the-synonym-tp4026381.html >>>>>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>> http://lucene.472066.n3.nabble.com/Can-a-field-with-defined-synonym-be-searched-without-the-synonym-tp4026381p4026405.html >>>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>> >>>> -- >>>> Walter Underwood >>>> wun...@wunderwood.org >>>> >>>> >>>> >>>> >> >> -- >> Walter Underwood >> wun...@wunderwood.org >> >> >> >
