[
https://issues.apache.org/jira/browse/SOLR-14254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035577#comment-17035577
]
Adrien Grand commented on SOLR-14254:
-------------------------------------
I don't have any objections to changing the name but I don't think FST50 is
inappropriate. We don't always rename index formats when we change them, for
instance we kept the name Lucene60PointsFormat when we added selective indexing.
Maybe we could make the situation less trappy by rejecting upgrading indices
that have non-default formats in Solr, or changing non-default formats to use a
version number that is computed using the current Lucene version.
> Index backcompat break between 8.3.1 and 8.4.1
> ----------------------------------------------
>
> Key: SOLR-14254
> URL: https://issues.apache.org/jira/browse/SOLR-14254
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Jason Gerlowski
> Priority: Major
>
> I believe I found a backcompat break between 8.4.1 and 8.3.1.
> I encountered this when a Solr 8.3.1 cluster was upgraded to 8.4.1. On 8.4.
> nodes, several collections had cores fail to come up with
> {{CorruptIndexException}}:
> {code}
> 2020-02-10 20:58:26.136 ERROR
> (coreContainerWorkExecutor-2-thread-1-processing-n:192.168.1.194:8983_solr) [
> ] o.a.s.c.CoreContainer Error waiting for SolrCore to be loaded on startup
> => org.apache.sol
> r.common.SolrException: Unable to create core
> [testbackcompat_shard1_replica_n1]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1313)
> org.apache.solr.common.SolrException: Unable to create core
> [testbackcompat_shard1_replica_n1]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1313)
> ~[?:?]
> at
> org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:788)
> ~[?:?]
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:202)
> ~[metrics-core-4.0.5.jar:4.0.5]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> ~[?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ~[?:?]
> at java.lang.Thread.run(Thread.java:834) [?:?]
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072) ~[?:?]
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901) ~[?:?]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1292)
> ~[?:?]
> ... 7 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2182)
> ~[?:?]
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2302)
> ~[?:?]
> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1132)
> ~[?:?]
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1013) ~[?:?]
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901) ~[?:?]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1292)
> ~[?:?]
> ... 7 more
> Caused by: org.apache.lucene.index.CorruptIndexException: codec mismatch:
> actual codec=Lucene50PostingsWriterDoc vs expected
> codec=Lucene84PostingsWriterDoc
> (resource=MMapIndexInput(path="/Users/jasongerlowski/run/solrdata/data/testbackcompat_shard1_replica_n1/data/index/_0_FST50_0.doc"))
> at
> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:208)
> ~[?:?]
> at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:198)
> ~[?:?]
> at
> org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:255) ~[?:?]
> at
> org.apache.lucene.codecs.lucene84.Lucene84PostingsReader.<init>(Lucene84PostingsReader.java:82)
> ~[?:?]
> at
> org.apache.lucene.codecs.memory.FSTPostingsFormat.fieldsProducer(FSTPostingsFormat.java:66)
> ~[?:?]
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:315)
> ~[?:?]
> at
> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:395)
> ~[?:?]
> at
> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:114)
> ~[?:?]
> at
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:84) ~[?:?]
> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> ~[?:?]
> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> ~[?:?]
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> ~[?:?]
> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526) ~[?:?]
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:116) ~[?:?]
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:92) ~[?:?]
> at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:39)
> ~[?:?]
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2146)
> ~[?:?]
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2302)
> ~[?:?]
> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1132)
> ~[?:?]
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1013) ~[?:?]
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901) ~[?:?]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1292)
> ~[?:?]
> ... 7 more
> {code}
> Only some cores were affected. After some digging, we noticed that these
> cores were unique in that they had a field set up to use Solr's Text Tagger.
> Not the use of a non-default postings-format (FST50).
> {code}
> <fieldType class="solr.TextField" name="tagger" omitNorms="true"
> omitTermFreqAndPositions="true" postingsFormat="FST50">
> <analyzer type="index">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.EnglishPossessiveFilterFactory"/>
> <filter class="solr.ASCIIFoldingFilterFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.ConcatenateGraphFilterFactory"
> preservePositionIncrements="false"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.EnglishPossessiveFilterFactory"/>
> <filter class="solr.ASCIIFoldingFilterFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> </analyzer>
> </fieldType>
> <field name="tagger_field" type="tagger" indexed="true" stored="true"
> />
> {code}
> I've since been able to reproduce the problem locally with the following
> steps:
> # Start Solr 8.3.1 with a data directory /some/data/dir
> # Modify the default configset (or create a new configset) by adding the
> fieldType/field above to your schema.
> # Index a document that contains a "tagger_field" value and commit.
> # Stop Solr 8.3.1
> # Start Solr 8.4.1 pointed at the /some/data/dir as its data directory.
> At this point, the core should be down and you should see the
> CorruptIndexException in your logs.
> This _seems_ related to LUCENE-9027, though I don't know enough about how
> Lucene handles ensuring support for older indices to know exactly what
> happened there.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
