This looks related to the new changes around schema validation. Lucene now requires a field to either be absent from a document or be indexed with the exact same options (index options, points dimensions, norms, doc values type, etc.) as already indexed documents that also have this field.
However it's a bug that Lucene fails to open an index that was legal in Lucene 8. Can you file a JIRA issue? On Mon, Dec 13, 2021 at 4:23 PM Ian Lea <ian....@gmail.com> wrote: > > Hi > > > We have a long-standing index with some mandatory fields and some optional > fields that has been through multiple lucene upgrades without a full > rebuild and on testing out an upgrade from version 8.11.0 to 9.0.0, when > open an IndexWriter we are hitting the exception > > Exception in thread "main" java.lang.IllegalArgumentException: cannot > change field "language" from index options=NONE to inconsistent index > options=DOCS > at > org.apache.lucene.index.FieldInfo.verifySameIndexOptions(FieldInfo.java:245) > at > org.apache.lucene.index.FieldInfos$FieldNumbers.verifySameSchema(FieldInfos.java:421) > at > org.apache.lucene.index.FieldInfos$FieldNumbers.addOrGet(FieldInfos.java:357) > at > org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:1263) > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1116) > > Where language is one of our optional fields. > > Presumably this is at least somewhat related to "Index options can no > longer be changed dynamically" as mentioned at > https://lucene.apache.org/core/9_0_0/MIGRATE.html although it fails before > our code attempts to update the index, and we are not trying to change any > index options. > > Adding some displays to IndexWriter and FieldInfos and logging rather than > throwing the exception I see > > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=DOCS > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=NONE > language curr=NONE, other=DOCS > language curr=NONE, other=DOCS > language curr=NONE, other=DOCS > language curr=NONE, other=DOCS > language curr=NONE, other=DOCS > language curr=NONE, other=DOCS > language curr=NONE, other=DOCS > language curr=NONE, other=DOCS > > where there is one line per segment. It logs the exception whenever > other=DOCS. Subset with segment info: > > segment _x8(8.2.0):c31753/-1:[diagnostics={timestamp=1565623850605, > lucene.version=8.2.0, java.vm.version=11.0.3+7, java.version=11.0.3, > mergeMaxNumSegments=-1, os.version=3.1.0-1.2-desktop, > java.vendor=AdoptOpenJDK, source=merge, os.arch=amd64, mergeFactor=10, > java.runtime.version=11.0.3+7, > os=Linux}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}] > > language curr=NONE, other=NONE > > segment _y9(8.7.0):c43531/-1:[diagnostics={timestamp=1604597581562, > lucene.version=8.7.0, java.vm.version=11.0.3+7, java.version=11.0.3, > mergeMaxNumSegments=-1, os.version=3.1.0-1.2-desktop, > java.vendor=AdoptOpenJDK, source=merge, os.arch=amd64, mergeFactor=10, > java.runtime.version=11.0.3+7, > os=Linux}]:[attributes={Lucene87StoredFieldsFormat.mode=BEST_SPEED}] > > language curr=NONE, other=DOCS > > NOT throwing java.lang.IllegalArgumentException: cannot change field > "language" from index options=NONE to inconsistent index options=DOCS > > > Some variation on an old-fashioned not set versus not present bug perhaps? > > > -- > Ian. -- Adrien --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
