Question about scoring normalisation

2005-11-05 Thread Karl Koch
Hello all, I am wondering how many of you actually work with own scoring mechanism (overwriting Lucenes standard scoring) and how many of you do work on how to normalise this score. I would like to add a second score on top of Lucenes TF/IDF score. The resulting score is most likely higher then

Re: Question about scoring normalisation

2005-11-05 Thread Chris Lamprecht
Lucene just takes the highest score returned, and divides all scores by this max_score. So max_score / max_score = 1.0, and voila. On 11/5/05, Karl Koch <[EMAIL PROTECTED]> wrote: > Hello all, > > I am wondering how many of you actually work with own scoring mechanism > (overwriting Lucenes stand

Re: Question about scoring normalisation

2005-11-05 Thread Sameer Shisodia
if so the top score should always be 1.0. Isn't so. Or does boosting multiple individual fields wreck that ? sameer On 11/6/05, Chris Lamprecht <[EMAIL PROTECTED]> wrote: > Lucene just takes the highest score returned, and divides all scores > by this max_score. So max_score / max_score = 1.0, an

Re: Question about scoring normalisation

2005-11-06 Thread Otis Gospodnetic
> if so the top score should always be 1.0. Isn't so. Perhaps you are right. Can you send some code that shows this, preferably writen as a JUnit test and attached to a JIRA issues? > Or does boosting multiple individual fields wreck that ? I didn't think so, but I could be wrong. Otis > same

Re: Question about scoring normalisation

2005-11-06 Thread Yonik Seeley
On 11/5/05, Sameer Shisodia <[EMAIL PROTECTED]> wrote: > if so the top score should always be 1.0. Isn't so. > Or does boosting multiple individual fields wreck that ? > sameer The top score is scaled back to 1.0 *only* if it's greater than 1.0 So hits with scores of 4.0,2.0 will be normalized to

Re: Question about scoring normalisation

2005-11-06 Thread Karl Koch
che Nachricht --- > Von: Ira Goldstein <[EMAIL PROTECTED]> > An: "Karl Koch" <[EMAIL PROTECTED]> > Betreff: Re: Question about scoring normalisation > Datum: Sun, 06 Nov 2005 08:08:59 -0500 > > Karl -- > Hi. I've been thinking about adding a pivoted norm

Re: Question about scoring normalisation

2005-11-07 Thread Ian Soboroff
"Karl Koch" <[EMAIL PROTECTED]> writes: > I am not sure if I know exactly what pivoted normalisation is. I can tell > you what I do, in the meantime I will have a look to your paper and I hope > that we can discuss this issue further. Sort answer on pivoted document length normalization. You'll