[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334283#comment-15334283 ] ASF subversion and git services commented on LUCENE-7132: - Commit 707bcc9b3bdae7b2bb2b9a7d9e30e1aa348587cb in lucene-solr's branch refs/heads/branch_5x from [~steve_rowe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=707bcc9b ] LUCENE-7132: BooleanQuery sometimes assigned the wrong score when ranges of documents had only one clause matching while other ranges had more than one clause matchng (Cherry-picked from commit 5dfaf0392fcd3b7e4b529dce0cd1035b766880a7) > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334284#comment-15334284 ] ASF subversion and git services commented on LUCENE-7132: - Commit 9f513d5569db42fe10b6580e69a754b7aa05f596 in lucene-solr's branch refs/heads/branch_6_0 from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9f513d5 ] LUCENE-7132: BooleanQuery sometimes assigned the wrong score when ranges of documents had only one clause matching while other ranges had more than one clause matchng > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334281#comment-15334281 ] ASF subversion and git services commented on LUCENE-7132: - Commit 77844e2591235bfc1944e901922f876c1d43c264 in lucene-solr's branch refs/heads/branch_5_5 from [~steve_rowe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=77844e2 ] LUCENE-7132: BooleanQuery sometimes assigned the wrong score when ranges of documents had only one clause matching while other ranges had more than one clause matchng (Cherry-picked from commit 5dfaf0392fcd3b7e4b529dce0cd1035b766880a7) > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334282#comment-15334282 ] ASF subversion and git services commented on LUCENE-7132: - Commit 4f6bddefe3310e0361c9b57fd522781d82c89bb8 in lucene-solr's branch refs/heads/branch_5_5 from [~steve_rowe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4f6bdde ] LUCENE-7132: Add 5.5.2 CHANGES entry > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316568#comment-15316568 ] ASF subversion and git services commented on LUCENE-7132: - Commit 5dfaf0392fcd3b7e4b529dce0cd1035b766880a7 in lucene-solr's branch refs/heads/branch_6x from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5dfaf03 ] LUCENE-7132: BooleanQuery sometimes assigned the wrong score when ranges of documents had only one clause matching while other ranges had more than one clause matchng > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316561#comment-15316561 ] ASF subversion and git services commented on LUCENE-7132: - Commit c8570ed821654cdce5f92ae17d06a21f242524e2 in lucene-solr's branch refs/heads/master from Mike McCandless [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c8570ed ] LUCENE-7132: BooleanQuery sometimes assigned the wrong score when ranges of documents had only one clause matching while other ranges had more than one clause matchng > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316481#comment-15316481 ] Michael McCandless commented on LUCENE-7132: Thanks [~jpountz] and [~hossman], I'll commit the last patch. > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316427#comment-15316427 ] Adrien Grand commented on LUCENE-7132: -- Thanks [~mikemccand] and Hoss for digging this sneaky bug! +1 to the patch bq. I think this bug is serious enough that we should be sure to get it into 6.1.0 ... I marked blocker. +1 > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315859#comment-15315859
]
Michael McCandless commented on LUCENE-7132:
I also confirmed that if I revert the fix in {{BooleanScorer.java}} that
{{TestBoolean2}} does fail on some seeds ... thanks [~hossman_luc...@fucit.org]!
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
>Priority: Blocker
> Fix For: 6.1, master (7.0)
>
> Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch,
> LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch,
> SOLR-8884.patch, SOLR-8884.patch, debug.xml
>
>
> Some of the folks
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes
> explain's score can be different than the score requested by fields
> parameter. Interestingly, Explain's scores would create a different ranking
> than the original result list. This is something users experience, but it
> cannot be re-produced deterministically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315857#comment-15315857 ] Michael McCandless commented on LUCENE-7132: I think this bug is serious enough that we should be sure to get it into 6.1.0 ... I marked blocker. > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe >Priority: Blocker > Fix For: 6.1, master (7.0) > > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315854#comment-15315854
]
Michael McCandless commented on LUCENE-7132:
Hmm, {{testQueries10}} changes the similarity of {{searcher}} temporarily, but
fails to also change {{singleSegmentSearcher}}'s similarity ... when I fix
that, then this seed passes.
But then on beasting I see other new failures ... digging.
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
> Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch,
> LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, SOLR-8884.patch,
> SOLR-8884.patch, debug.xml
>
>
> Some of the folks
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes
> explain's score can be different than the score requested by fields
> parameter. Interestingly, Explain's scores would create a different ranking
> than the original result list. This is something users experience, but it
> cannot be re-produced deterministically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315851#comment-15315851 ] Michael McCandless commented on LUCENE-7132: Thanks [~hossman], I can reproduce that failure with your patch ... I'll dig. It is baffling :) > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, SOLR-8884.patch, > SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314505#comment-15314505
]
Hoss Man commented on LUCENE-7132:
--
{noformat}
10:21 <@hoss:#lucene-dev> mikemccand: ping?
...
10:44 <@mikemccand:#lucene-dev> hoss: here
10:45 <@hoss:#lucene-dev> oh yeah ... just writting up a jira response ... i
think you generated your patch just using "git diff" so it missed
the "new" test files?
10:45 <@hoss:#lucene-dev> i've got a unified patch i'm about to post, so we
have both the fix andthe tests that reliably demonstrate the
problem
10:45 <@mikemccand:#lucene-dev> oh yeah sorry i did!
10:45 <@mikemccand:#lucene-dev> ++ thanks
10:46 <@hoss:#lucene-dev> no worries ... what i really wanted to ping you about
was writting a better test
10:46 <@hoss:#lucene-dev> right now that test shouldn't be committed as is --
data fro ma user i'm certain we don't have rights to
10:46 <@mikemccand:#lucene-dev> ahh yeah that should be fun :)
10:46 <@mikemccand:#lucene-dev> yeah i saw the comment about that ...
10:46 <@hoss:#lucene-dev> i'm wondering if you could give me some pointers on
the hueristics that lead to this optimizatio, so i can try to
write a tighter test case that hits it?
10:46 <@hoss:#lucene-dev> (to prevent a regression)
10:46 <@mikemccand:#lucene-dev> ok lemme try
10:47 <@mikemccand:#lucene-dev> right, we need a test
10:47 <@mikemccand:#lucene-dev> you need a 2 clause BQ
10:47 <@mikemccand:#lucene-dev> where a document with docID 0 -- 2047 matches
only one term
10:47 <@mikemccand:#lucene-dev> and then another docID > 2047 matches two terms
10:47 <@mikemccand:#lucene-dev> in that case the 2nd document should get the
wrong (disagrees w/ explain) score i think
10:48 <@hoss:#lucene-dev> docID 0 .. as in literally docID 0 in teh index? ...
it was that magical?
10:48 <@mikemccand:#lucene-dev> yes!
10:48 <@mikemccand:#lucene-dev> BS1 scores in windows of 2048 documents
10:48 <@hoss:#lucene-dev> holy fuck that's an edge case
10:48 <@mikemccand:#lucene-dev> the bug is that if 1 window uses an
"optimization" because only 1 clauses matches ...
10:48 <@mikemccand:#lucene-dev> then that optimization messes up the state
10:48 <@mikemccand:#lucene-dev> and subsequent windows get the wrong scores
10:48 <@mikemccand:#lucene-dev> yeah serious edge case :)
10:48 <@mikemccand:#lucene-dev> i'm glad you pushed on this :)
10:48 <@mikemccand:#lucene-dev> thanks
10:49 <@hoss:#lucene-dev> oh ... so like, doc ID 2049 matching, but no other
doc matches until after 2048 * 2 would also hit this bug?
10:49 <@hoss:#lucene-dev> actually ... it sounds like any doc matching as long
as it's the only doc in it's window, and then another doc in a
alater window?
10:49 <@mikemccand:#lucene-dev> right!
10:50 <@mikemccand:#lucene-dev> (where that later window's doc had more than 1
clause matching)
10:50 <@hoss:#lucene-dev> ok .. so really, we just need more tests with lots of
docs, so that we force matches across the windows ... because
2048 is hardcoded, not somethign we can randomize to
small values via LTC
10:50 <@mikemccand:#lucene-dev> yeah ..
10:51 <@hoss:#lucene-dev> hmmm... why did forceMerge change things then?
10:51 <@hoss:#lucene-dev> with no deletions wy did the windows change?
10:51 <@mikemccand:#lucene-dev> hmmm i'm not sure?
10:51 <@mikemccand:#lucene-dev> the forceMerge is crazy: the index already had
one segment
10:51 <@mikemccand:#lucene-dev> at least for your first seed
10:52 <@mikemccand:#lucene-dev> yet forceMerge DID run, because CFS wanted to
change
10:52 <@mikemccand:#lucene-dev> but this should not have altered the docID order
10:52 <@mikemccand:#lucene-dev> so yeah i can't explain why forceMerge
"matters" here
10:53 <@hoss:#lucene-dev> and yet - if it wasn't for the forceMerge, the only
indication of the bug would be that the Explanations don't match
-- unless we hardcoded scores in a test, which is
hard for randomized data
10:53 <@hoss:#lucene-dev> nee impossible
10:54 <@mikemccand:#lucene-dev> yes
10:57 <@hoss:#lucene-dev> weird maybe there's another factor to the
optimization we need to consider? ... i'll let you ponder while i try
to figure out a test bsaed on what we know :)
10:57 <@mikemccand:#lucene-dev> LOL ok
...
11:04 <@hoss:#lucene-dev> mikemccand: BTW, you mind if i transcribe this conv
to jira so i don't lose it?
11:04 <@mikemccand:#lucene-dev> ++ great
{noformat}
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL:
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314149#comment-15314149 ] Michael McCandless commented on LUCENE-7132: [~rcmuir] pointed out that LUCENE-7138 is likely the same bug ... I'll confirm. > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > LUCENE-7132.patch, SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313941#comment-15313941
]
Michael McCandless commented on LUCENE-7132:
A bit more progress ... it looks like somehow {{BooleanScorer}} gets confused
and is collecting a window with more than one sub-scorer matching, yet failed
to tell the top collector to reset back to the fake scorer (it's still on a
boosted scorer, I think from a prior window that had only one matching sub,
incorrectly applying a 0. boost).
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
> Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch,
> SOLR-8884.patch, SOLR-8884.patch, debug.xml
>
>
> Some of the folks
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes
> explain's score can be different than the score requested by fields
> parameter. Interestingly, Explain's scores would create a different ranking
> than the original result list. This is something users experience, but it
> cannot be re-produced deterministically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313899#comment-15313899
]
Michael McCandless commented on LUCENE-7132:
OK if I put back BQ's bulkScorer, but then use {{new IndexSearcher}} instead of
{{newSearcher}} the first seed above gets further along: the scores agree
before/after forceMerge, but the explanation is still incorrect...
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
> Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch,
> SOLR-8884.patch, SOLR-8884.patch, debug.xml
>
>
> Some of the folks
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes
> explain's score can be different than the score requested by fields
> parameter. Interestingly, Explain's scores would create a different ranking
> than the original result list. This is something users experience, but it
> cannot be re-produced deterministically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313895#comment-15313895
]
Michael McCandless commented on LUCENE-7132:
Hmm seems like something is wrong w/ BQ's bulk scorer, because if I take it out
and just use the default:
{noformat}
@@ -361,6 +363,9 @@ final class BooleanWeight extends Weight {
@Override
public BulkScorer bulkScorer(LeafReaderContext context) throws IOException {
+// nocommit
+return super.bulkScorer(context);
+/*
final BulkScorer bulkScorer = booleanScorer(context);
if (bulkScorer != null) {
// bulk scoring is applicable, use it
@@ -369,6 +374,7 @@ final class BooleanWeight extends Weight {
// use a Scorer-based impl (BS2)
return super.bulkScorer(context);
}
+*/
}
@Override
{noformat}
Then at least the first seed above passes...
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
> Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch,
> SOLR-8884.patch, SOLR-8884.patch, debug.xml
>
>
> Some of the folks
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes
> explain's score can be different than the score requested by fields
> parameter. Interestingly, Explain's scores would create a different ranking
> than the original result list. This is something users experience, but it
> cannot be re-produced deterministically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313885#comment-15313885 ] Michael McCandless commented on LUCENE-7132: Thanks [~hossman] I'll dig. > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, > SOLR-8884.patch, SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15225118#comment-15225118
]
Uwe Schindler commented on LUCENE-7132:
---
With LUCENE-7174 committed (update of randomizedtesting framework), your test
produces the following failure:
{noformat}
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene60),
sim=RandomSimilarity(queryNorm=false,coord=no): {field=DFR I(n)B2, alt=DFR
I(ne)B1}, locale=sr-Latn, timezone=America/St_Vincent
[junit4] 2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_72
(64-bit)/cpus=8,threads=1,free=166881504,total=261095424
[junit4] 2> NOTE: All tests run in this JVM: [TestSimpleExplanations]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestSimpleExplanations -Dtests.seed=74B719CE50C8168A
-Dtests.slow=true -Dtests.locale=sr-Latn -Dtests.timezone=America/St_Vincent
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s | TestSimpleExplanations (suite) <<<
[junit4]> Throwable #1: junit.framework.AssertionFailedError: Clean up
static fields (in @AfterClass?) and null them, your test still has references
to classes of which the sizes cannot be measured due to security restrictions
or Java 9 module encapsulation:
[junit4]> - protected static org.apache.lucene.index.RandomIndexWriter
org.apache.lucene.search.BaseExplanationTestCase.writer
[junit4]>at
__randomizedtesting.SeedInfo.seed([74B719CE50C8168A]:0)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4]> Caused by: java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "accessClassInPackage.sun.nio.fs")
[junit4]>at
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
[junit4]>at
java.security.AccessController.checkPermission(AccessController.java:884)
[junit4]>at
java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
[junit4]>at
java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1564)
[junit4]>at java.lang.Class.checkPackageAccess(Class.java:2372)
[junit4]>at java.lang.Class.checkMemberAccess(Class.java:2351)
[junit4]>at java.lang.Class.getDeclaredFields(Class.java:1915)
[junit4]>at java.security.AccessController.doPrivileged(Native
Method)
[junit4]>... 10 more
[junit4] Completed [1/1 (1!)] in 2.90s, 69 tests, 1 failure <<< FAILURES!
{noformat}
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
> Attachments: LUCENE-7132.patch, LUCENE-7132.patch, SOLR-8884.patch,
> SOLR-8884.patch, debug.xml
>
>
> Some of the folks
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes
> explain's score can be different than the score requested by fields
> parameter. Interestingly, Explain's scores would create a different ranking
> than the original result list. This is something users experience, but it
> cannot be re-produced deterministically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223206#comment-15223206
]
Uwe Schindler commented on LUCENE-7132:
---
I added https://github.com/randomizedtesting/randomizedtesting/issues/227 to
make this easier to debug. The stack trace is not very helpful to figure out
that the "bad" guy was the non-nulled {{writer}} field.
> BooleanQuery scores can be diff for same docs+sim when using coord (disagree
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
> Attachments: LUCENE-7132.patch, LUCENE-7132.patch, SOLR-8884.patch,
> SOLR-8884.patch, debug.xml
>
>
> Some of the folks
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes
> explain's score can be different than the score requested by fields
> parameter. Interestingly, Explain's scores would create a different ranking
> than the original result list. This is something users experience, but it
> cannot be re-produced deterministically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223008#comment-15223008 ] Uwe Schindler commented on LUCENE-7132: --- You have to null the writer fiel in BaseExplanationTrstCase like searcher in afterClass. It's shown how to do for the other fields there :) > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, SOLR-8884.patch, > SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)
[ https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223004#comment-15223004 ] Uwe Schindler commented on LUCENE-7132: --- bq. Dawid Weiss, Uwe Schindler - any idea what i did to causes this weird AccessControlException in TestSimpleExplanations? The problem is caused by the static leak detector (same issue like the fix I posted to Solr). The leak detetctor measures static fields and their heap usage. Unfortunately this fails for non-public classes (e.g. implementations of Path). So be sure to null all static fields on afterClass(). In Java 9 this fails more often, so be sure to always null out static non-final fields that point to "more complex" and internal Java objects. > BooleanQuery scores can be diff for same docs+sim when using coord (disagree > with Explanation which doesn't change) > --- > > Key: LUCENE-7132 > URL: https://issues.apache.org/jira/browse/LUCENE-7132 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 5.5 >Reporter: Ahmet Arslan >Assignee: Steve Rowe > Attachments: LUCENE-7132.patch, LUCENE-7132.patch, SOLR-8884.patch, > SOLR-8884.patch, debug.xml > > > Some of the folks > [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes > explain's score can be different than the score requested by fields > parameter. Interestingly, Explain's scores would create a different ranking > than the original result list. This is something users experience, but it > cannot be re-produced deterministically. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
