Sent to your personal email address.
-- Ian. On Wed, Sep 10, 2014 at 12:36 PM, Robert Muir <rcm...@gmail.com> wrote: > Ian, this looks terrible, thanks for reporting this. Is there any > possible way I could have a copy of that "working" index to make it > easier to reproduce? > > On Wed, Sep 10, 2014 at 7:01 AM, Ian Lea <ian....@gmail.com> wrote: >> Hi >> >> >> On running a quick test after a handful of minor code changes to deal >> with 4.10 deprecations, a program that updates an existing index >> failed with >> >> Exception in thread "main" java.lang.IllegalStateException: cannot >> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) >> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607) >> >> and along the way did something to the index to make it unusable. >> >> Digging a bit deeper and working on a different old test index that >> was lying around, and taking a backup first this time, this is >> reproducible. >> >> The working index: >> >> total 1036 >> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt >> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx >> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm >> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq >> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim >> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip >> -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si >> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2 >> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen >> >> and output from 4.10 CheckIndex >> >> Opening index @ index/ >> >> Segments file=segments_2 numSegments=1 version=4.0.0.2 format= >> 1 of 1: name=_0 docCount=15730 >> version=4.0.0.2 >> codec=Lucene40 >> compound=false >> numFiles=7 >> size (MB)=0.987 >> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, >> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 >> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle >> Corporation} >> no deletions >> test: open reader.........OK >> test: check integrity.....OK >> test: check live docs.....OK >> test: fields..............OK [13 fields] >> test: field norms.........OK [0 fields] >> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs >> pairs; 139382 tokens] >> test: stored fields.......OK [15730 total field count; avg 1 fields per >> doc] >> test: term vectors........OK [0 total vector count; avg 0 >> term/freq vector fields per doc] >> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 >> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET] >> >> No problems were detected with this index. >> >> >> Now run this little program >> >> public static void main(final String[] _args) throws Exception { >> File index = new File(_args[0]); >> IndexWriterConfig iwcfg = new IndexWriterConfig(Version.LUCENE_4_10_0, >> new StandardAnalyzer()); >> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND); >> Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index)); >> IndexWriter iw = new IndexWriter(d, iwcfg); >> Document doc1 = new Document(); >> doc1.add(new StringField("type", "test", Field.Store.NO)); >> iw.addDocument(doc1); >> iw.close(); >> } >> >> and it fails with >> >> Exception in thread "main" java.lang.IllegalStateException: cannot >> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40) >> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607) >> at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524) >> at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:1017) >> at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549) >> at >> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3062) >> at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169) >> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915) >> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986) >> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956) >> at t.main(t.java:25) >> >> and when run CheckIndex again get >> >> >> Opening index @ index/ >> >> ERROR: could not read any segments file in directory >> java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si >> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) >> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) >> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) >> at >> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:176) >> at java.nio.channels.FileChannel.open(FileChannel.java:287) >> at java.nio.channels.FileChannel.open(FileChannel.java:334) >> at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:196) >> at >> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:52) >> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362) >> at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458) >> at >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:913) >> at >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:759) >> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454) >> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414) >> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096) >> >> which is true >> >> total 1032 >> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt >> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx >> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm >> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq >> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim >> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip >> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2 >> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen >> >> >> I don't recall the origins of this index but it may well have been >> created in the distant past and been upgraded, explicitly or >> automatically, along the way. Although evidently not for a while. >> >> >> Running the same test with lucene 4.9.0 and minimal mods to the >> program runs to successful completion. Here's the CheckIndex output: >> >> >> Opening index @ index-4.9.updated/ >> >> Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format= >> 1 of 2: name=_0 docCount=15730 >> codec=Lucene40 >> compound=false >> numFiles=7 >> size (MB)=0.987 >> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop, >> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06 >> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle >> Corporation} >> no deletions >> test: open reader.........OK >> test: check integrity.....OK >> test: check live docs.....OK >> test: fields..............OK [13 fields] >> test: field norms.........OK [0 fields] >> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs >> pairs; 139382 tokens] >> test: stored fields.......OK [15730 total field count; avg 1 fields per >> doc] >> test: term vectors........OK [0 total vector count; avg 0 >> term/freq vector fields per doc] >> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 >> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET] >> >> 2 of 2: name=_1 docCount=1 >> codec=Lucene49 >> compound=true >> numFiles=3 >> size (MB)=0.001 >> diagnostics = {timestamp=1410281698360, os=Linux, >> os.version=3.1.0-1.2-desktop, source=flush, lucene.version=4.9.0 >> 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64, >> java.version=1.7.0_10, java.vendor=Oracle Corporation} >> no deletions >> test: open reader.........OK >> test: check integrity.....OK >> test: check live docs.....OK >> test: fields..............OK [1 fields] >> test: field norms.........OK [0 fields] >> test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens] >> test: stored fields.......OK [0 total field count; avg 0 fields per doc] >> test: term vectors........OK [0 total vector count; avg 0 >> term/freq vector fields per doc] >> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0 >> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET] >> >> No problems were detected with this index. >> >> >> >> >> -- >> Ian. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org