This could be down to IDF ie "Lucane" is ranked higher because it is rarer
despite having worse edit distance.
This is arguably a bug.
See http://issues.apache.org/jira/browse/LUCENE-329 which discusses this. You
could try subclass QueryParser and override newFuzzyQuery to return
FuzzyLikeThisQuery (found in "contrib/queries")
Cheers
Mark
----- Original Message ----
From: stefcl <stefatw...@gmail.com>
To: java-user@lucene.apache.org
Sent: Mon, 15 February, 2010 14:13:52
Subject: Strange Fuzzyquery results scoring when using a low minimal distance
Hello,
I'm using Lucene v3.
Please consider the following spellings
Lucene
Lucéne
lucéne
Lucane
Lucen
When searching for "lucéne" among those words using a FuzzyQuery (with 0.5
edit distance), results show :
1. Lucene 1.0259752
2. Lucane 1.0259752
3. Lucéne 0.95660806
4. lucéne 0.95660806
5. Lucen 0.30779266
#4 is an exact match, why does it receive a lower score than "Lucane" which
contains one incorrect letter?
Also, if you raise min similarity a bit higher (0.6 of above), everything
becomes normal :
1. Lucéne 1.0438477
2. lucéne 1.0438477
3. Lucene 0.97959816
4. Lucane 0.97959816
Any idea?
Thanks in advance...
The code I use :
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws IOException,
ParseException
{
StandardAnalyzer analyzer = new
StandardAnalyzer(Version.LUCENE_CURRENT);
// TODO code application logic here
Directory index = new RAMDirectory();
IndexWriter w = new IndexWriter(index, analyzer, true,
IndexWriter.MaxFieldLength.UNLIMITED);
addDoc(w, "Lucene");
addDoc(w, "Lucéne");
addDoc(w, "lucéne");
addDoc(w, "Lucane");
addDoc(w, "Lucen");
w.close();
FuzzyQuery q = new FuzzyQuery( new Term("title", "lucéne") , 0.5f
);
// 3. search
IndexSearcher searcher = new IndexSearcher(index);
TopDocs collector = searcher.search(q, 10);
ScoreDoc[] hits = collector.scoreDocs;
// 4. display results
System.out.println("Found " + hits.length + " hits.");
for(int i = 0 ; i < hits.length; i++)
{
Document d = searcher.doc(hits[i].doc);
System.out.println((i + 1) + ". " + d.get("title") + " " +
hits[i].score );
}
// searcher can only be closed when there
// is no need to access the documents any more.
searcher.close();
}
private static void addDoc(IndexWriter w, String value) throws
IOException
{
Document doc = new Document();
doc.add(new Field("title", value, Field.Store.YES,
Field.Index.ANALYZED));
w.addDocument(doc);
}
--
View this message in context:
http://old.nabble.com/Strange-Fuzzyquery-results-scoring-when-using-a-low-minimal-distance-tp27594371p27594371.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
