vstrout opened a new issue, #12067:
URL: https://github.com/apache/lucene/issues/12067
### Description
After upgrading from Lucene 9.3.0 to Lucene 9.4.2 the index search with
sorting by description throws the following exception:
Caused by: java.lang.IllegalStateException: Term [77 73 64 66 6a 66 73 67 73
20 61 64 6b 66 64 6a 68 74 67 64 67 20 61 64 6b 66 64 6a 68 74 67 64 67 20 72
65 74 72 65 72 74 65 20] exists in doc values but not in the terms index
at
org.apache.lucene.search.comparators.TermOrdValComparator$CompetitiveIterator.init(TermOrdValComparator.java:582)
at
org.apache.lucene.search.comparators.TermOrdValComparator$CompetitiveIterator.update(TermOrdValComparator.java:553)
at
org.apache.lucene.search.comparators.TermOrdValComparator$TermOrdValLeafComparator.updateCompetitiveIterator(TermOrdValComparator.java:457)
at
org.apache.lucene.search.comparators.TermOrdValComparator$TermOrdValLeafComparator.setHitsThresholdReached(TermOrdValComparator.java:284)
at
org.apache.lucene.search.TopFieldCollector$TopFieldLeafCollector.countHit(TopFieldCollector.java:86)
at
org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector$1.collect(TopFieldCollector.java:202)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:305)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:247)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:744)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:662)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:656)
at
org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:636)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:553)
I wrote a small java function that reproduces the issue. Increasing maxRows
value there to above 200 somehow resolves the problem.
public static void test() throws Exception {
String name = "description";
String content = "content";
File dir = new File("C:\\test");
MMapDirectory directory = new MMapDirectory(dir.toPath());
Analyzer analyzer = new
StandardAnalyzer(EnglishAnalyzer.ENGLISH_STOP_WORDS_SET);
boolean createIndex = true; // must be changed to false after
calling this function first time
if (createIndex) {
String[] values = {
"anshduvfv ",
"dhisdefihfhg ",
"Afasdfasdf ",
"Retrerte ",
"bnbssdfgfg ",
"wrgfhhjg ",
"jfhtvg ",
"fhdhfdsads ",
"Wsdfjfsgs ",
"adkfdjhtgdg ",
};
Random random = new Random();
IndexWriterConfig config = new
IndexWriterConfig(analyzer);
config.setOpenMode(OpenMode.CREATE_OR_APPEND);
IndexWriter writer = new IndexWriter(directory, config);
for (int i = 0; i < 110; i++) {
for (int j = 0; j < 10; j++) {
String value = values[j] +
values[random.nextInt(10)] + values[random.nextInt(10)] +
values[random.nextInt(10)];
Document doc = new Document();
doc.add(new TextField(content, value,
Field.Store.NO));
doc.add(new StringField(name, value,
Field.Store.YES));
doc.add(new SortedDocValuesField(name,
new BytesRef(value.toLowerCase()))); // case-insensitive sorting
writer.addDocument(doc);
}
}
writer.close();
}
int maxRows = 100;
String request = "*:*";
DirectoryReader reader = DirectoryReader.open(directory);
IndexSearcher searcher = new IndexSearcher(reader);
QueryParser parser = new QueryParser(content, analyzer);
parser.setSplitOnWhitespace(true);
parser.setAllowLeadingWildcard(false);
Query query = parser.parse(request);
Sort sort = new Sort(new SortField(name, Type.STRING, true));
TopDocs docs = searcher.search(query, maxRows, sort);
reader.close();
}
### Version and environment details
Upgraded from Lucene 9.3.0 to Lucene 9.4.2. Using
lucene-backward-codecs-9.4.2.jar
OS: MS Windows 11
Java: jdk-11.0.14
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
