I know one person who, I am 99.99999% sure, will have an answer to this - Bob Carpenter. I'm BCC-ing Bob here. Bob, the question is whether we (Apache Lucene) can quote (on Lucene's site) a person who run Lucene precision/recall quality tests on some TEC corpus and reported results. Do you know?
I think the answer is positive. Look at page 7 (of only 8) of this report http://trec.nist.gov/pubs/trec13/papers/unorthtexas.qa.pdf (used LingPipe with TREC data). Otis ----- Original Message ---- From: Grant Ingersoll <[EMAIL PROTECTED]> To: java-dev@lucene.apache.org Sent: Monday, June 25, 2007 8:48:03 PM Subject: Re: search quality - assessment & improvements On Jun 25, 2007, at 2:19 PM, Doron Cohen wrote: >> IANAL and I didn't read the link, but I think people publish their >> MAP scores, etc. all the time on TREC data. I think it implies that >> you obtained the data through legal means. > > So you're saying that if person "X" got the TREC data legally, we > can have > in our (say) benchmarks age, something like: > (*) Person "X" reports the following TREC measures... > And anyone discussing his TREC results with Lucene in Lucene's mailing > lists does this under the list assumption that he got the TREC data > legally. Sounds practical to me, at least to start with. It seems reasonable, but I am not an authority. One way to do it, is to look for TREC citations in papers. By the way, the link in your orig. paper is password protected. A search for TREC precision recall on Yahoo! yields papers that discuss past runs of TREC that were not published as part of TREC. Of course, that doesn't make it right, but my gut feeling is it is not a big deal assuming you came about the data legally. In general, people publish their precision and recall scores given a collection. Without the name of the collection, the scores are meaningless. -Grant --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
