Dear friends
hi
I want to use Lucene to calculate Precision and Recall.
I did these steps:
1- made some index files. to do this I used indexer code and indexed txt
files which exist in this path "C:/inn" (there are 4 txt files in this
folder) and take them in "outt" folder by setting the indexpath to "C:/outt"
in the Indexer code.
2- then creat a package called lia.benchmark and a class inside it which is
called "PrecisionRecall" and add externaljars (rightclick>Java build path
>add external jars) and added Lucene-benchmark-.3.2.0jar and
Lucene-core-3.3.0jar
3- set the topicsfile path in code to"C:/lia2e/src/lia/benchmark/topics.txt"
and
qrelsfile to "C:/lia2e/src/lia/benchmark/qrels.txt" and dir to "C:/outt".
here is code:
package lia.benchmark;
import java.io.File;
import java.io.PrintWriter;
import java.io.BufferedReader;
import java.io.FileReader;
import org.apache.lucene.search.*;
import org.apache.lucene.store.*;
import org.apache.lucene.benchmark.quality.*;
import org.apache.lucene.benchmark.quality.utils.*;
import org.apache.lucene.benchmark.quality.trec.*;
public class PrecisionRecall {
public static void main(String[] args) throws Throwable {
File topicsFile = new File("C:/lia2e/src/lia/benchmark/topics.txt");
File qrelsFile = new File("C:/lia2e/src/lia/benchmark/qrels.txt");
Directory dir = FSDirectory.open(new File("C:/outt"));
IndexSearcher searcher = new IndexSearcher(dir, true);
String docNameField = "filename";
PrintWriter logger = new PrintWriter(System.out, true);
TrecTopicsReader qReader = new TrecTopicsReader();
QualityQuery qqs[] = qReader.readQueries(
new BufferedReader(new FileReader(topicsFile)));
Judge judge = new TrecJudge(new BufferedReader(
new FileReader(qrelsFile)));
judge.validateData(qqs, logger);
QualityQueryParser qqParser = new SimpleQQParser("title", "contents");
QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher,
docNameField);
SubmissionReport submitLog = null;
QualityStats stats[] = qrun.execute(judge,
submitLog, logger);
QualityStats avg = QualityStats.average(stats);
avg.log("SUMMARY",2,logger, " ");
dir.close();
}
}
4- and initialized qrels and topics. in documents folder(C:\inn) I have 4
txt files which 2 of them is relevance to my query ( query is apple)
so I filled qrels and topics .
the qrels file like this:
<top>
<num> Number: 0
<title> apple
<desc> Description:
<narr> Narrative:
</top>
and topics file like this:
0 0 789.txt 1
0 0 101.txt 1
I tried also the Path format namely for example "C:\inn\789.txt" instead
of "789.txt"
but results are zero :
the resuluts are these:
0 - contents:apple
0 Stats:
Search Seconds: 0.016
DocName Seconds: 0.000
Num Points: 2.000
Num Good Points: 0.000
Max Good Points: 2.000
Average Precision: 0.000
MRR: 0.000
Recall: 0.000
Precision At 1: 0.000
SUMMARY
Search Seconds: 0.016
DocName Seconds: 0.000
Num Points: 2.000
Num Good Points: 0.000
Max Good Points: 2.000
Average Precision: 0.000
MRR: 0.000
Recall: 0.000
Precision At 1: 0.000
can you tell me what is wrong with me?
I really need to know why results are zero.
please help me.
thanks in advanc.
vahideh
