Sent to your personal email address.
--
Ian.
On Wed, Sep 10, 2014 at 12:36 PM, Robert Muir <rcm...@gmail.com> wrote:
> Ian, this looks terrible, thanks for reporting this. Is there any
> possible way I could have a copy of that "working" index to make it
> easier to reproduce?
>
> On Wed, Sep 10, 2014 at 7:01 AM, Ian Lea <ian....@gmail.com> wrote:
>> Hi
>>
>>
>> On running a quick test after a handful of minor code changes to deal
>> with 4.10 deprecations, a program that updates an existing index
>> failed with
>>
>> Exception in thread "main" java.lang.IllegalStateException: cannot
>> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
>> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>>
>> and along the way did something to the index to make it unusable.
>>
>> Digging a bit deeper and working on a different old test index that
>> was lying around, and taking a backup first this time, this is
>> reproducible.
>>
>> The working index:
>>
>> total 1036
>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>> -rw-r--r-- 1 tril users 311 Jan 18 2013 _0.si
>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>
>> and output from 4.10 CheckIndex
>>
>> Opening index @ index/
>>
>> Segments file=segments_2 numSegments=1 version=4.0.0.2 format=
>> 1 of 1: name=_0 docCount=15730
>> version=4.0.0.2
>> codec=Lucene40
>> compound=false
>> numFiles=7
>> size (MB)=0.987
>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
>> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
>> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
>> Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [13 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
>> pairs; 139382 tokens]
>> test: stored fields.......OK [15730 total field count; avg 1 fields per
>> doc]
>> test: term vectors........OK [0 total vector count; avg 0
>> term/freq vector fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
>> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> No problems were detected with this index.
>>
>>
>> Now run this little program
>>
>> public static void main(final String[] _args) throws Exception {
>> File index = new File(_args[0]);
>> IndexWriterConfig iwcfg = new IndexWriterConfig(Version.LUCENE_4_10_0,
>> new StandardAnalyzer());
>> iwcfg.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
>> Directory d = FSDirectory.open(index, new SimpleFSLockFactory(index));
>> IndexWriter iw = new IndexWriter(d, iwcfg);
>> Document doc1 = new Document();
>> doc1.add(new StringField("type", "test", Field.Store.NO));
>> iw.addDocument(doc1);
>> iw.close();
>> }
>>
>> and it fails with
>>
>> Exception in thread "main" java.lang.IllegalStateException: cannot
>> write 3x SegmentInfo unless codec is Lucene3x (got: Lucene40)
>> at org.apache.lucene.index.SegmentInfos.write3xInfo(SegmentInfos.java:607)
>> at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:524)
>> at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:1017)
>> at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4549)
>> at
>> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3062)
>> at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3169)
>> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:915)
>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:986)
>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:956)
>> at t.main(t.java:25)
>>
>> and when run CheckIndex again get
>>
>>
>> Opening index @ index/
>>
>> ERROR: could not read any segments file in directory
>> java.nio.file.NoSuchFileException: /tmp/lucene/index/_0.si
>> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>> at
>> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:176)
>> at java.nio.channels.FileChannel.open(FileChannel.java:287)
>> at java.nio.channels.FileChannel.open(FileChannel.java:334)
>> at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:196)
>> at
>> org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:52)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:362)
>> at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:458)
>> at
>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:913)
>> at
>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:759)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:454)
>> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:414)
>> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2096)
>>
>> which is true
>>
>> total 1032
>> -rw-r--r-- 1 tril users 165291 Jan 18 2013 _0.fdt
>> -rw-r--r-- 1 tril users 125874 Jan 18 2013 _0.fdx
>> -rw-r--r-- 1 tril users 1119 Jan 18 2013 _0.fnm
>> -rw-r--r-- 1 tril users 378015 Jan 18 2013 _0_Lucene40_0.frq
>> -rw-r--r-- 1 tril users 350628 Jan 18 2013 _0_Lucene40_0.tim
>> -rw-r--r-- 1 tril users 13988 Jan 18 2013 _0_Lucene40_0.tip
>> -rw-r--r-- 1 tril users 69 Jan 18 2013 segments_2
>> -rw-r--r-- 1 tril users 20 Jan 18 2013 segments.gen
>>
>>
>> I don't recall the origins of this index but it may well have been
>> created in the distant past and been upgraded, explicitly or
>> automatically, along the way. Although evidently not for a while.
>>
>>
>> Running the same test with lucene 4.9.0 and minimal mods to the
>> program runs to successful completion. Here's the CheckIndex output:
>>
>>
>> Opening index @ index-4.9.updated/
>>
>> Segments file=segments_3 numSegments=2 versions=[4.0.0.2 .. 4.9] format=
>> 1 of 2: name=_0 docCount=15730
>> codec=Lucene40
>> compound=false
>> numFiles=7
>> size (MB)=0.987
>> diagnostics = {os=Linux, os.version=3.1.0-1.2-desktop,
>> source=flush, lucene.version=4.0.0 1394950 - rmuir - 2012-10-06
>> 02:58:12, os.arch=amd64, java.version=1.7.0_10, java.vendor=Oracle
>> Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [13 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [53466 terms; 217447 terms/docs
>> pairs; 139382 tokens]
>> test: stored fields.......OK [15730 total field count; avg 1 fields per
>> doc]
>> test: term vectors........OK [0 total vector count; avg 0
>> term/freq vector fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
>> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> 2 of 2: name=_1 docCount=1
>> codec=Lucene49
>> compound=true
>> numFiles=3
>> size (MB)=0.001
>> diagnostics = {timestamp=1410281698360, os=Linux,
>> os.version=3.1.0-1.2-desktop, source=flush, lucene.version=4.9.0
>> 1604085 - rmuir - 2014-06-20 06:22:23, os.arch=amd64,
>> java.version=1.7.0_10, java.vendor=Oracle Corporation}
>> no deletions
>> test: open reader.........OK
>> test: check integrity.....OK
>> test: check live docs.....OK
>> test: fields..............OK [1 fields]
>> test: field norms.........OK [0 fields]
>> test: terms, freq, prox...OK [1 terms; 1 terms/docs pairs; 0 tokens]
>> test: stored fields.......OK [0 total field count; avg 0 fields per doc]
>> test: term vectors........OK [0 total vector count; avg 0
>> term/freq vector fields per doc]
>> test: docvalues...........OK [0 docvalues fields; 0 BINARY; 0
>> NUMERIC; 0 SORTED; 0 SORTED_NUMERIC; 0 SORTED_SET]
>>
>> No problems were detected with this index.
>>
>>
>>
>>
>> --
>> Ian.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
