Hey y'all,
We ran into a strange postings merge error in production and I am not sure
if its how Elasticsearch is handling the merges or if it's a bug in Lucene.
The FST compiler reaches the "merge" line when merging some segments:
if (lastInput.length() == input.length && prefixLenPlus1 == 1 +
input.length) {
// same input more than 1 time in a row, mapping to
// multiple outputs
lastNode.output = fst.outputs.merge(lastNode.output, output);
However, the "outputs" provided by Lucene90BlockTreeTermsWriter is
ByteSequenceOutputs, which does not override merge, and thus throws an
unsupported operation exception.
Given this, it seems like it should be impossible to reach the
"Outputs.merge" path when merging with the Lucene90BlockTreeTermsWriter,
but somehow it did.
Any ideas on where I should look? Does this warrant a bug in Lucene?
The trace is:
```
Caused by:
org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper:
unsupported_operation_exception: null
at org.apache.lucene.util.fst.Outputs.merge(Outputs.java:95)
~[lucene-core-9.11.1.jar:?]
at org.apache.lucene.util.fst.FSTCompiler.add(FSTCompiler.java:936)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$PendingBlock.append(Lucene90BlockTreeTermsWriter.java:593)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$PendingBlock.compileIndex(Lucene90BlockTreeTermsWriter.java:562)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$TermsWriter.writeBlocks(Lucene90BlockTreeTermsWriter.java:776)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter$TermsWriter.finish(Lucene90BlockTreeTermsWriter.java:1163)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsWriter.write(Lucene90BlockTreeTermsWriter.java:402)
~[lucene-core-9.11.1.jar:?]
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:95)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:204)
~[lucene-core-9.11.1.jar:?]
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:211)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.index.SegmentMerger.mergeWithLogging(SegmentMerger.java:300)
~[lucene-core-9.11.1.jar:?]
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:139)
~[lucene-core-9.11.1.jar:?]
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5293)
~[lucene-core-9.11.1.jar:?]
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4761)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:6582)
~[lucene-core-9.11.1.jar:?]
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:660)
~[lucene-core-9.11.1.jar:?]
at
org.elasticsearch.index.engine.ElasticsearchConcurrentMergeScheduler.doMerge(ElasticsearchConcurrentMergeScheduler.java:134)
~[elasticsearch-8.15.0.jar:?]
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:721)
~[lucene-core-9.11.1.jar:?]
```
