We have still a problem with queries that rewrite depending on index contents - which was the reason for MTQ's deMorgan bug. If two MultiTermQueries rewrite to different queries on two shards, the scores are also not comparable, even with normalized idf. This does not affect WildCard&Co (because default to constant score), but e.g. Fuzzy will be very broken multi-sharded. MultiSearcher tried to prevent this by combining all rewritten queries into one - and was buggy here.
We reinvent MultiSearcher because of this (Mike's code in 3191 is a partly reincarnation of MultiSearcher), only the buggy combine is missing. ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Michael McCandless [mailto:[email protected]] > Sent: Friday, June 10, 2011 9:57 PM > To: [email protected] > Subject: Re: Distributed search capability > > On Fri, Jun 10, 2011 at 1:48 PM, Andrzej Bialecki <[email protected]> wrote: > > On 6/10/11 6:27 PM, Michael McCandless wrote: > >> > >> I'm actually working on something like this, basically a utility > >> method to merge N TopDocs into 1. I want to do this for grouping as > >> well to make it easy to do grouping across shards. > > > > Mike, > > > > The straightforward merge that is used in Solr suffers from > > incomparable scores (due to the lack of global IDF). See my slides from the > Buzzwords. > > Since we can handle global IDF in local searchers more easily that in > > Solr then we can reuse that DfCache trick from MultiSearcher. > > This is cool stuff Andrzej!! > > But, my patch (LUCENE-3191) is aiming for the lower-level problem of just > the mechanics of merging multiple TopDocs.... ie, something "above" will > have to handle "properly" setting scores of the incoming TopDocs (if in fact > the search sorts by score). > > Mike McCandless > > http://blog.mikemccandless.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] For additional > commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
