[ http://issues.apache.org/jira/browse/LUCENE-736?page=all ]
Doron Cohen updated LUCENE-736:
-------------------------------
Attachment: sloppy_phrase_java.patch.txt
perf-search-new.log
perf-search-orig.log
Attached sloppy_phrase_java.patch.txt is fixing the failing new tests.
This also includes the skipTo() bug from issue 697.
The fix does not guarantee that document A B C B A would score "A B C"~4 and "C
B A"~4 the same.
It does that for "B C"~2 and "C B"~2.
This is because a general fix for that (at least the one that I devised) would
be too expensive.
Although this is an interesting case, I'd like to think it is not an important
one.
This fix comes with a performance cost: about 15% degradation in CPU activity
of sloppy phrase scoring, as the attcahed perf logs show.
Here is the summary of these tests:
.......Operation..........runCnt...recsPerRun.....rec/s..elapsedSec
Orig:..SearchSameRdr_3000......4.........3000.....216.1.......55.52
New:...SearchSameRdr_3000......4.........3000.....187.8.......63.91
I think that in a real life scenario - real index, real documents, real queries
- this extra CPU will be shaded by IO, but I also belive we should refrain from
slowing down search, so, unhappy with this degradation (anyone would:-), I
would look for a other ways to fix this - ideas are welcome.
Perf test was done using the task benchmark framework (see issue 675), The logs
show also the queries that were searched.
All tests pass with new code.
> Sloppy Phrase Scoring Misbehavior
> ---------------------------------
>
> Key: LUCENE-736
> URL: http://issues.apache.org/jira/browse/LUCENE-736
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Reporter: Doron Cohen
> Assigned To: Doron Cohen
> Priority: Minor
> Attachments: perf-search-new.log, perf-search-orig.log,
> sloppy_phrase_java.patch.txt, sloppy_phrase_tests.patch.txt
>
>
> This is an extension of https://issues.apache.org/jira/browse/LUCENE-697
> In addition to abnormalities Yonik pointed out in 697, there seem to be other
> issues with slopy phrase search and scoring.
> 1) A phrase with a repeated word would be detected in a document although it
> is not there.
> I.e. document = A B D C E , query = "B C B" would not find this document (as
> expected), but query "B C B"~2 would find it.
> I think that no matter how large the slop is, this document should not be a
> match.
> 2) A document containing both orders of a query, symmetrically, would score
> differently for the queru and for its reveresed form.
> I.e. document = A B C B A would score differently for queries "B C"~2 and "C
> B"~2, although it is symmetric to both.
> I will attach test cases that show both these problems and the one reported
> by Yonik in 697.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
