Hi,
we are currently testing Solr 9.7 and experiencing an error we have not seen
before with SolR 9.6.1 and we think the problem might occur in the underlying
lucene code basis:
ERROR o.a.s.h.RequestHandlerBase Server exception =>
at
org.apache.lucene.search.TopFieldCollector.populateScores(TopFieldCollector.java:478)
java.lang.IllegalArgumentException: Doc id 48567944 doesn't match the query
at
org.apache.lucene.search.TopFieldCollector.populateScores(TopFieldCollector.java:478)
~[?:?]
at
org.apache.solr.search.SolrIndexSearcher.populateScoresIfNeeded(SolrIndexSearcher.java:1766)
~[?:?]
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1955)
~[?:?]
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1729)
~[?:?]
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:726)
~[?:?]
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:721)
~[?:?]
at
org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1690)
~[?:?]
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:432)
~[?:?]
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:456)
~[?:?]
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)
~[?:?]
We index the embeddings as nested fields and can reproduce the error with the
following code:
We are able to reproduce the error using the same query. It seems to occur in
approximately 5% of all vector queries.
We have one server running Solr 9.7 and three servers running Solr 9.6.1, all
working on the same frozen index.
Only the Solr 9.7 server encounters the issue. We can rule out Java 21 and the
corresponding optimizations or the new multithreading parameter as the root
cause of the problem.
The index contains the document referenced in the error message.
String q = "{!knn f=vector topK=20}[0.031046804...";
SolrQuery sq = new SolrQuery("{!cache=false}" + q);
sq.addField("score"); // no error without the score field
sq.setRows(14); // Defect document must be included in result
sq.setSort("ID", ORDER.asc); // Order is not important, but ID is. No error
e.g. with score
final QueryRequest r = new QueryRequest(sq, METHOD.POST);
SolrClient solrClient = SolRConnector.createServer(server);
QueryResponse response = r.process(solrClient);
Is there any additional information we can provide to help resolve this error?
Best regards
Andreas Moll
