[ https://issues.apache.org/jira/browse/LUCENE-329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Harwood updated LUCENE-329: -------------------------------- Attachment: LUCENE-329.patch New patch addressing this long-standing bug. Addresses the all-or-nothing choices of today where the default is a (poor) use of all IDF factors or a sub-optimal alternative of using a rewrite method with no IDF. The patch includes: 1) A new default FuzzyQuery rewrite method that balances IDF better 2) Unit tests for single and multi-query behaviours Additionally, this document offers more analysis based on quality tests on a slightly larger set of data not included here: https://docs.google.com/document/d/1KXhbUpD5GFyzNqfk3nocODOo7Upgpd5tmUQp4-OPwiM/edit#heading=h.2e8gdmdqf2m5 > Fuzzy query scoring issues > -------------------------- > > Key: LUCENE-329 > URL: https://issues.apache.org/jira/browse/LUCENE-329 > Project: Lucene - Core > Issue Type: Bug > Components: core/search > Affects Versions: 1.2 > Environment: Operating System: All > Platform: All > Reporter: Mark Harwood > Assignee: Mark Harwood > Priority: Minor > Fix For: 3.1, 4.0-ALPHA > > Attachments: ASF.LICENSE.NOT.GRANTED--patch.txt, LUCENE-329.patch > > > Queries which automatically produce multiple terms (wildcard, range, prefix, > fuzzy etc)currently suffer from two problems: > 1) Scores for matching documents are significantly smaller than term queries > because of the volume of terms introduced (A match on query Foo~ is 0.1 > whereas a match on query Foo is 1). > 2) The rarer forms of expanded terms are favoured over those of more common > forms because of the IDF. When using Fuzzy queries for example, rare mis- > spellings typically appear in results before the more common correct > spellings. > I will attach a patch that corrects the issues identified above by > 1) Overriding Similarity.coord to counteract the downplaying of scores > introduced by expanding terms. > 2) Taking the IDF factor of the most common form of expanded terms as the > basis of scoring all other expanded terms. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org