Re: Lucene's Mean Average Precision

2008-05-15 Thread Dave Kor
I haven't participated in TREC for the past 2 years, so I am wonder which TREC track were you comparing your results against? The last time I checked, Lucene's score for the Terabyte track wasn't wonderful, but it was still pretty decent. Bear in mind that Lucene uses the plain old vanilla TF-IDF

Re: Lucene's Mean Average Precision

2008-05-04 Thread Otis Gospodnetic
lr - Nutch - Original Message > From: DanaWhite <[EMAIL PROTECTED]> > To: java-user@lucene.apache.org > Sent: Sunday, May 4, 2008 7:28:57 PM > Subject: Re: Lucene's Mean Average Precision > > > I arrived at this MAP by modifying IndexFiles to use a

Re: Lucene's Mean Average Precision

2008-05-04 Thread Grant Ingersoll
On May 4, 2008, at 7:28 PM, DanaWhite wrote: I arrived at this MAP by modifying IndexFiles to use a StopAnalyzer and work in a way that was acceptable for TReC files. The SearchFiles was modified to use a StopAnalyzer and output data in a trec_eval suitable format. Trec_eval reports abou

Re: Lucene's Mean Average Precision

2008-05-04 Thread DanaWhite
I arrived at this MAP by modifying IndexFiles to use a StopAnalyzer and work in a way that was acceptable for TReC files. The SearchFiles was modified to use a StopAnalyzer and output data in a trec_eval suitable format. Trec_eval reports about 11% at this setting. I am not competing in TReC I

Re: Lucene's Mean Average Precision

2008-05-04 Thread Grant Ingersoll
How did you arrive at that MAP? What analyzers, etc.? So much of search depends on your choices during indexing and querying, etc. There is some work by the IBM Haifa people up on the Wiki, so that would be one place to check. Another question is what is your end goal is in doing a TREC

Lucene's Mean Average Precision

2008-05-04 Thread DanaWhite
Hello all, I have been doing some evaluation of Lucene on a TReC collection and get a rather disappointing mean average precision (MAP) of 11%. Other sources seem to report a MAP of about 20%. So I am here to ask all of you who have done some Lucene evaluation - what did you find Lucene's MAP t