[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270197#comment-17270197 ] Christine Poerschke commented on SOLR-15071: Thanks [~dsmiley] for the PR reviews! As per above the changes have been committed to master, branch_8x and branch_8_8 branches. I'll keep this ticket open until it's known if the fix will be included in 8.8.0 or not. If not then the CHANGES.txt entry will need to be moved out of the 8.8.0 section and the "Fix Version" attributes of the JIRA be updated here. > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.serv
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270174#comment-17270174 ] ASF subversion and git services commented on SOLR-15071: Commit e2e375c397701b6bd3d5c55f7e48bc3dc465f9ee in lucene-solr's branch refs/heads/branch_8_8 from Florin Babes [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e2e375c ] SOLR-15071: add TestEdisMaxSolrFeature.testEdisMaxSolrFeatureCustomMM() test case (#2201) * add test case for SOLR-15071 * add temporary @Ignore to be removed when the fix is committed Co-authored-by: Florin Babes Co-authored-by: Christine Poerschke > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.ecli
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270175#comment-17270175 ] ASF subversion and git services commented on SOLR-15071: Commit e2e375c397701b6bd3d5c55f7e48bc3dc465f9ee in lucene-solr's branch refs/heads/branch_8_8 from Florin Babes [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e2e375c ] SOLR-15071: add TestEdisMaxSolrFeature.testEdisMaxSolrFeatureCustomMM() test case (#2201) * add test case for SOLR-15071 * add temporary @Ignore to be removed when the fix is committed Co-authored-by: Florin Babes Co-authored-by: Christine Poerschke > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.ecli
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270176#comment-17270176 ] ASF subversion and git services commented on SOLR-15071: Commit 3046bcf608fb6bdd6fe8fcd5fd7213103e802c9d in lucene-solr's branch refs/heads/branch_8_8 from Christine Poerschke [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3046bcf ] SOLR-15071: Fix ArrayIndexOutOfBoundsException in contrib/ltr SolrFeatureScorer (#2196) > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > )
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270163#comment-17270163 ] ASF subversion and git services commented on SOLR-15071: Commit db95a292725c480b71e9ec8c05a54b60d32d55f6 in lucene-solr's branch refs/heads/branch_8x from Christine Poerschke [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=db95a29 ] SOLR-15071: Fix ArrayIndexOutOfBoundsException in contrib/ltr SolrFeatureScorer (#2196) > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > ) >
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270126#comment-17270126 ] ASF subversion and git services commented on SOLR-15071: Commit 32e95ddb3f2542fbe2cb116ac58453182149f07b in lucene-solr's branch refs/heads/master from Christine Poerschke [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=32e95dd ] SOLR-15071: Fix ArrayIndexOutOfBoundsException in contrib/ltr SolrFeatureScorer (#2196) > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > ) > at
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264968#comment-17264968 ] ASF subversion and git services commented on SOLR-15071: Commit ed89eae3e7ad6f952ebda0d22f5aeb268894aa5d in lucene-solr's branch refs/heads/branch_8x from Florin Babes [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ed89eae ] SOLR-15071: add TestEdisMaxSolrFeature.testEdisMaxSolrFeatureCustomMM() test case (#2201) * add test case for SOLR-15071 * add temporary @Ignore to be removed when the fix is committed Co-authored-by: Florin Babes Co-authored-by: Christine Poerschke > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.j
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264969#comment-17264969 ] ASF subversion and git services commented on SOLR-15071: Commit ed89eae3e7ad6f952ebda0d22f5aeb268894aa5d in lucene-solr's branch refs/heads/branch_8x from Florin Babes [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ed89eae ] SOLR-15071: add TestEdisMaxSolrFeature.testEdisMaxSolrFeatureCustomMM() test case (#2201) * add test case for SOLR-15071 * add temporary @Ignore to be removed when the fix is committed Co-authored-by: Florin Babes Co-authored-by: Christine Poerschke > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.j
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264774#comment-17264774 ] ASF subversion and git services commented on SOLR-15071: Commit f285f02c8968c30a149ba3c3462a42403fd4d069 in lucene-solr's branch refs/heads/master from Florin Babes [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f285f02 ] SOLR-15071: add TestEdisMaxSolrFeature.testEdisMaxSolrFeatureCustomMM() test case (#2201) * add test case for SOLR-15071 * add temporary @Ignore to be removed when the fix is committed Co-authored-by: Florin Babes Co-authored-by: Christine Poerschke > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jett
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264773#comment-17264773 ] ASF subversion and git services commented on SOLR-15071: Commit f285f02c8968c30a149ba3c3462a42403fd4d069 in lucene-solr's branch refs/heads/master from Florin Babes [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f285f02 ] SOLR-15071: add TestEdisMaxSolrFeature.testEdisMaxSolrFeatureCustomMM() test case (#2201) * add test case for SOLR-15071 * add temporary @Ignore to be removed when the fix is committed Co-authored-by: Florin Babes Co-authored-by: Christine Poerschke > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 1h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jett
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264619#comment-17264619 ] Florin Babes commented on SOLR-15071: - Hello [~cpoerschke]. I've created a test to reproduce the issue here [https://github.com/apache/lucene-solr/pull/2201.] > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 50m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > ) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215 > ) > at > org.eclipse.
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264219#comment-17264219 ] Florin Babes commented on SOLR-15071: - Hello [~cpoerschke], I will try to create a test for that issue. I will try to finish it next week because tomorrow I will go on vacation for a few days. > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 0.5h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > ) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263587#comment-17263587 ] Christine Poerschke commented on SOLR-15071: {quote}... why does SFS not (potentially) have two-phase iterator support? Looking at the code, it seems it simply delegates to the Scorer it wraps, ... {quote} Yes, that's what I meant by "potentially" i.e. if the wrapped scorer supports it then SFS claims to support it too. If the wrapped scorer does not support it then SFS also does not support it. {quote}... if that Scorer supports it, shouldn't SFS likewise? ... {quote} In principle yes but [observationally|https://github.com/apache/lucene-solr/pull/2196#issuecomment-758599976] in {{MinShouldMatchSumScorer}} at least after the use of the two-phase iterator the {{score}} method can no longer be called. Haven't yet looked further into the exact details. Prior to 8.6 i.e. [https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.5.2/solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java#L269-L294] SFS did not have two-phase iterator support. I would suggest to scope of this ticket here to _re-instate the non-support_ of two-phase iteration and the _addition of two-phase support_ would be a separate future enhancement. bq. ... A real unit test would be ideal. Are there obstacles make this difficult? I agree. [https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/contrib/ltr/src/test/org/apache/solr/ltr/feature/TestEdisMaxSolrFeature.java] is an existing test that could be extended or a new test similar to it could be created. [~florin.babes] and/or [~dizzu333] would you be interested in having a go at that? > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 0.5h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263564#comment-17263564 ] David Smiley commented on SOLR-15071: - {quote}the SolrFeatureScorer inadvertently claimed to potentially have two-phase iteration support when it does not have it{quote} Can you elaborate on that... why does SFS not (potentially) have two-phase iterator support? Looking at the code, it seems it simply delegates to the Scorer it wraps, so if that Scorer supports it, shouldn't SFS likewise? The only thing I can think of that would cause this to be false would be if SFS did some sort of filtering, but I don't see it. A real unit test would be ideal. Are there obstacles make this difficult? > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 0.5h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.ecl
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263283#comment-17263283 ] Florin Babes commented on SOLR-15071: - Thanks for your help [~cpoerschke]! > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 0.5h > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > ) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215 > ) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > at > org.eclipse.
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263214#comment-17263214 ] Ovidiu Mihalcea commented on SOLR-15071: Thank you for your help, [~cpoerschke]. I've compiled and tested with your fix and everything is ok now. > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Assignee: Christine Poerschke >Priority: Major > Labels: ltr > Fix For: 8.8 > > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 20m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > ) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215 > ) > at > org.eclipse.jetty.serv
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262947#comment-17262947 ] Christine Poerschke commented on SOLR-15071: * The {{freq >= minShouldMatch}} assertion in the {{updateFreq}} method fails. ** https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/lucene/core/src/java/org/apache/lucene/search/MinShouldMatchSumScorer.java#L303 * This is because in the {{setDocAndFreq}} method the {{freq}} is re-computed and somehow that doesn't quite work out. ** https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/lucene/core/src/java/org/apache/lucene/search/MinShouldMatchSumScorer.java#L303 * {{setDocAndFreq}} is called by the {{TwoPhaseIterator}} methods. ** https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/lucene/core/src/java/org/apache/lucene/search/MinShouldMatchSumScorer.java#L135 * {{twoPhaseIterator()}} is an optional {{Scorer}} method, it defaults to {{null}} i.e. two-pharse iteration is not supported. ** https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/lucene/core/src/java/org/apache/lucene/search/Scorer.java#L74-L91 * With the SOLR-14378 and SOLR-14364 changes the {{SolrFeatureScorer}} inadvertently claimed to have two-phase iteration support when it does not have it. ** https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java#L233 Proposed fix: https://github.com/apache/lucene-solr/pull/2196 > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > Attachments: featurestore+model+sample_documents.zip > > Time Spent: 10m > Remaining Estimate: 0h > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Serv
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262916#comment-17262916 ] Christine Poerschke commented on SOLR-15071: Well, this is a truely interesting bug! I think I've figured out what the issue is and how it got broken and how to fix it. Easy fix but with long explanation to go with it, will continue after dinner :-) > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > Attachments: featurestore+model+sample_documents.zip > > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580 > ) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215 > ) > at > org.eclipse.jetty.server.handler.Scope
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262764#comment-17262764 ] Ovidiu Mihalcea commented on SOLR-15071: At first I thought I've managed to reproduce it with even a single document: {{*curl -XGET 'http://localhost:8983/solr/techproducts/select?fl=id%2C%5Bfeatures%20store%3Ddev_2020_08_26%20efi.term%3D%22husa%20cu%20tastatura%20tableta%2010%20inch%22%5D&q=husa%20cu%20tastat*}} {{*ura%20tableta%2010%20inch&fq=id:934521240'*}} And it worked, even after a restart (so no caches involved, I believe). -To sum up, the problem occurs when trying to extract a feature with the following params:- -*{{"q": "\{!dismax qf=name mm=3}${term}"}}*- -from the document with these fields:- -*{{{ "id":"934521240", "name":"tableta magnetica abc quercetti" }}}*- -where $term is:- -*{{husa cu tastatura tableta 10 inch}}*- But if I try adding only this document, or this document and another subset of the sample documents in the index, (_and this is the weird part_) the problem no longer occurs. So the problem seems to occur even with only one document returned in the request (fq=id:X), but only if other (_which ones?!_) documents are present, which logically makes me believe the request also touches some other documents, even though it shouldn't. > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > Attachments: featurestore+model+sample_documents.zip > > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) >
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262684#comment-17262684 ] Christine Poerschke commented on SOLR-15071: Great to hear the issue is reproducible with the techproducts example and the addition of some of your documents. Thanks [~dizzu333] for sharing the exact steps and the files to work with! I see that there are 10,000 documents and would be curious if reducing that somehow would lead us to further insights. Either indexing fewer documents or indexing all the 10,000 documents but excluding them from the search e.g. via {{fq=id:(123 OR 456 OR 789)}} clauses. Perhaps you've already tried that, I see there's {{rows=500}} on the query rather than the default {{rows=10}}? > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > Attachments: featurestore+model+sample_documents.zip > > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at org.eclipse.jetty.s
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262657#comment-17262657 ] Ovidiu Mihalcea commented on SOLR-15071: Hello, Maybe someone who recently went through the code related to this could help us identify the issue? [~dsmiley], if you have the time, could you also take a look at this bug and maybe help us identify the issue? It would be very appreciated. Thank you! Here are the exact reproduction steps we are using to trigger the bug (files: [^featurestore+model+sample_documents.zip] ): # start and index techproducts {{*$ /opt/solr/bin/solr -e techproducts -Dsolr.ltr.enabled=true*}} # upload feature store *{{$ curl -XPUT 'http://localhost:8983/solr/techproducts/schema/feature-store' --data-binary "@./feature-store.minimal.json" -H 'Content-type:application/json'}}* # upload model *{{$ curl -XPUT 'http://localhost:8983/solr/techproducts/schema/model-store' --data-binary "@./model.minimal.json" -H 'Content-type:application/json'}}* # index sample documents *{{$ curl -XPOST 'http://localhost:8983/solr/techproducts/update?wt=json&commitWithin=1&overwrite=true' --data-binary "@./sample_documents.json" -H 'Content-type:application/json'}}* # query for ‘husa cu tastatura tableta 10 inch’ *{{$ curl -XGET 'http://localhost:8983/solr/techproducts/select?fl=id%2C%5Bfeatures%20store%3Ddev_2020_08_26%20efi.term%3D"husa%20cu%20tastatura%20tableta%2010%20inch"%5D&q=husa%20cu%20tastatura%20tableta%2010%20inch&rows=500}}* > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > Attachments: featurestore+model+sample_documents.zip > > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(Se
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262458#comment-17262458 ] Florin Babes commented on SOLR-15071: - Thanks for the hints [~cpoerschke]. We've managed to reproduce the issue using the techproducts example and indexing some of our documents. The mm.autoRelax didn't help, same error. Same with using fq in features. We tried to debug the issue using intellij remote debug and we found the following difference. In 8.6.3 is called this method org/apache/lucene/search/MinShouldMatchSumScorer.java:updateFreq and is not called in 8.5.2. We've tried to find the main cause but without succes. The above method leads here org/apache/lucene/search/DisiPriorityQueue.java:add and at this line heap[size] = entry; the exception is triggered. It looks like the heap is not initialized with enough space for all terms. > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233 > ) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:130 > 0) > at > org.eclipse.jetty.server.handler.Scop
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260716#comment-17260716 ] Christine Poerschke commented on SOLR-15071: {quote}... we tend to believe it's because of the following changes: https://issues.apache.org/jira/browse/SOLR-14364 as they're the only major changes related to LTR which were introduced in Solr 8.6.0. ... {quote} Yes, that's a plausible hypothesis. SOLR-14364 (which supersedes [~slivotov]'s SOLR-12698) relates to the "no fq" scenario and the SolrFeature features above don't have an fq query, so that fits the picture. It's imaginable that the LTR feature computation code makes assumptions which are no longer met in certain scenarios after the change. The hypothesis could be tested further e.g. * via the addition of some sort of {{fq}} in the SolrFeature parameters, perhaps {{\*:\*}} would be suitable or otherwise an actual {{fq}}. * within a locally built Solr, observing the error behaviour after the changes but not in the commits before then. > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.common.SolrException", > "root-error-class","java.lang.ArrayIndexOutOfBoundsException"], > "msg":"java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds > for length 2", > The stacktrace is : > org.apache.solr.common.SolrException: > java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2 > at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:154) > at > org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:159 > 9) > at > org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1413 > ) > at > org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:596) > at > org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryC > omponent.java:1513) > at > org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:403 > ) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler. > java:360) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java > :214) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav > a:1596) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235 > ) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:161 > 0) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHa
[jira] [Commented] (SOLR-15071) Bug on LTR when using solr 8.6.3 - index out of bounds DisiPriorityQueue.add(DisiPriorityQueue.java:102)
[ https://issues.apache.org/jira/browse/SOLR-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259985#comment-17259985 ] Christine Poerschke commented on SOLR-15071: Thanks [~florin.babes] for creating this ticket as a continuation of the Solr user mailing list thread! bq. ... We've narrowed our model to only two features and the problem always occurs (for some queries, not all) when we have a feature with a mm=1 and a feature with a mm>=3. The problem also occurs when we only do feature extraction and the problem seems to always occur on the feature with the bigger mm. The errors seem to be related to the size of the head DisiPriorityQueue created here: https://github.com/apache/lucene-solr/blob/branch_8_6/lucene/core/src/java/org/apache/lucene/search/MinShouldMatchSumScorer.java#L107 as the error changes as we change the mm for the second feature: ... That really helps narrow things down, great. Thinking out aloud about possible further investigative steps: The problem also happening during feature extraction should makes it easier to build a reproducible test case e.g. with the techproducts example used in the Solr Reference Guide https://lucene.apache.org/solr/guide/8_6/learning-to-rank.html#quick-start-with-ltr i.e. given the documents in the techproducts example, is it possible to craft the dummy feature-store and choose mm values and queries that cause the same error? {quote} ... We have the following query raw parameters: q=lg cx 4k oled 120 hz -> just of many examples term_dq=lg cx 4k oled 120 hz rq={!ltr model=model reRankDocs=1000 store=feature_store efi.term=${term_dq}} ... {quote} You mention that the problem occurs for some queries but not all and that it seems to always occur for the bigger mm. Could the queries or the {{mm}} be adjusted to 'ensure' that the problem occurs? Specifically * if we have (say) {{lg cx 4k oled 120 hz}} then that's 6 terms and so would mm=7 then 'guarantee' hitting of the error? * might stopword removal be a factor e.g. if {{lg cx 4k oled 120 hz}} is pruned to (say) {{lg cx 4k oled hz}} then with only 5 terms left could mm>=6 ever work? Not having used {{mm}} myself I had to lookup https://lucene.apache.org/solr/guide/8_6/the-dismax-query-parser.html#mm-minimum-should-match-parameter to learn a little about it and I also noticed https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.3/solr/solrj/src/java/org/apache/solr/common/params/DisMaxParams.java#L45-L48 as sounding promising i.e. if {{mm.autoRelax=true}} was used would that avoid the errors? {code} /** * If set to true, will try to reduce MM if tokens are removed from some clauses but not all */ public static String MM_AUTORELAX = "mm.autoRelax"; {code} If the error can be reproduced with the techproducts example included in the 8.6.3 release itself then a next step could be to build Solr locally -- https://github.com/apache/lucene-solr/tree/master#building-solr -- and using the techproducts in the locally built Solr the code behaviour could be examined more closely to figure out exactly what's going on and how it might be fixed. Does that kind of make sense? > Bug on LTR when using solr 8.6.3 - index out of bounds > DisiPriorityQueue.add(DisiPriorityQueue.java:102) > > > Key: SOLR-15071 > URL: https://issues.apache.org/jira/browse/SOLR-15071 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - LTR >Affects Versions: 8.6, 8.7 >Reporter: Florin Babes >Priority: Major > Labels: ltr > > Hello, > We are trying to update Solr from 8.3.1 to 8.6.3. On Solr 8.3.1 we are > using LTR in production using a MultipleAdditiveTrees model. On Solr 8.6.3 > we receive an error when we try to compute some SolrFeatures. We didn't > find any pattern of the queries that fail. > Example: > We have the following query raw parameters: > q=lg cx 4k oled 120 hz -> just of many examples > term_dq=lg cx 4k oled 120 hz > rq={!ltr model=model reRankDocs=1000 store=feature_store > efi.term=${term_dq}} > defType=edismax, > mm=2<75% > The features are something like this: > { > "name":"similarity_query_fileld_1", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_1 mm=1}${term}"}, > "store":"feature_store" > }, > { > "name":"similarity_query_field_2", > "class":"org.apache.solr.ltr.feature.SolrFeature", > "params":\{"q":"{!dismax qf=query_field_2 mm=5}${term}"}, > "store":"feature_store" > } > We are testing ~6300 production queries and for about 1% of them we receive > that following error message: > "metadata":[ > "error-class","org.apache.solr.comm