[jira] Updated: (LUCENE-1262) IndexOutOfBoundsException from FieldsReader after problem reading the index
[ https://issues.apache.org/jira/browse/LUCENE-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1262: --- Attachment: LUCENE-1262.patch Attached patch. All tests pass. I plan to commit in a day or so, to both trunk (2.4) and 2.3.X branch (2.3.2). I got the failure to happen with a standalone test case, added to TestFieldsReader. I found fixed the issue. It's in BufferedIndexReader's refill() method. The problem is that method changes bufferLength even if an exception is hit. This leaves incorrect bytes in the buffer such that a subsequent readByte will return the incorrect bytes. The fix is simple: use a local int newLength and only assign that to value to bufferLength if the readInternal() call succeeds. The test fails without the fix and passes with it. IndexOutOfBoundsException from FieldsReader after problem reading the index --- Key: LUCENE-1262 URL: https://issues.apache.org/jira/browse/LUCENE-1262 Project: Lucene - Java Issue Type: Bug Components: Index Affects Versions: 2.3.1 Reporter: Trejkaz Attachments: LUCENE-1262.patch, Test.java There is a situation where there is an IOException reading from Hits, and then the next time you get a NullPointerException instead of an IOException. Example stack traces: java.io.IOException: The specified network name is no longer available at java.io.RandomAccessFile.readBytes(Native Method) at java.io.RandomAccessFile.read(RandomAccessFile.java:322) at org.apache.lucene.store.FSIndexInput.readInternal(FSDirectory.java:536) at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:74) at org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:220) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:93) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:34) at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:57) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:88) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:344) at org.apache.lucene.index.IndexReader.document(IndexReader.java:368) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:84) at org.apache.lucene.search.Hits.doc(Hits.java:104) That error is fine. The problem is the next call to doc generates: java.lang.NullPointerException at org.apache.lucene.index.FieldsReader.getIndexType(FieldsReader.java:280) at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:216) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:101) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:344) at org.apache.lucene.index.IndexReader.document(IndexReader.java:368) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:84) at org.apache.lucene.search.Hits.doc(Hits.java:104) Presumably FieldsReader is caching partially-initialised data somewhere. I would normally expect the exact same IOException to be thrown for subsequent calls to the method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-1262) IndexOutOfBoundsException from FieldsReader after problem reading the index
[ https://issues.apache.org/jira/browse/LUCENE-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Trejkaz updated LUCENE-1262: Affects Version/s: (was: 2.1) 2.3.1 Summary: IndexOutOfBoundsException from FieldsReader after problem reading the index (was: NullPointerException from FieldsReader after problem reading the index) I managed to reproduce the problem as-is under version 2.2. For 2.3 the problem has changed -- instead of a NullPointerException it is now an IndexOutOfBoundsException: Exception in thread main java.lang.IndexOutOfBoundsException: Index: 52, Size: 34 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:260) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:154) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:659) at org.apache.lucene.index.IndexReader.document(IndexReader.java:525) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:92) at org.apache.lucene.search.Hits.doc(Hits.java:167) at Test.main(Test.java:24) Will attach my test program in a moment. IndexOutOfBoundsException from FieldsReader after problem reading the index --- Key: LUCENE-1262 URL: https://issues.apache.org/jira/browse/LUCENE-1262 Project: Lucene - Java Issue Type: Bug Components: Index Affects Versions: 2.3.1 Reporter: Trejkaz There is a situation where there is an IOException reading from Hits, and then the next time you get a NullPointerException instead of an IOException. Example stack traces: java.io.IOException: The specified network name is no longer available at java.io.RandomAccessFile.readBytes(Native Method) at java.io.RandomAccessFile.read(RandomAccessFile.java:322) at org.apache.lucene.store.FSIndexInput.readInternal(FSDirectory.java:536) at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:74) at org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:220) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:93) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:34) at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:57) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:88) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:344) at org.apache.lucene.index.IndexReader.document(IndexReader.java:368) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:84) at org.apache.lucene.search.Hits.doc(Hits.java:104) That error is fine. The problem is the next call to doc generates: java.lang.NullPointerException at org.apache.lucene.index.FieldsReader.getIndexType(FieldsReader.java:280) at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:216) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:101) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:344) at org.apache.lucene.index.IndexReader.document(IndexReader.java:368) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:84) at org.apache.lucene.search.Hits.doc(Hits.java:104) Presumably FieldsReader is caching partially-initialised data somewhere. I would normally expect the exact same IOException to be thrown for subsequent calls to the method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (LUCENE-1262) IndexOutOfBoundsException from FieldsReader after problem reading the index
[ https://issues.apache.org/jira/browse/LUCENE-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Trejkaz updated LUCENE-1262: Attachment: Test.java Attaching a test program to reproduce the problem under 2.3.1. It occurs approximately 1 in every 4 executions for any reasonably large text index (really small ones don't seem to do it so I couldn't attach a text index with it.) The number of fields may be related, looking at the IndexOutOfBoundsException numbers it seems that the indexes we have happen to have a large number of fields. IndexOutOfBoundsException from FieldsReader after problem reading the index --- Key: LUCENE-1262 URL: https://issues.apache.org/jira/browse/LUCENE-1262 Project: Lucene - Java Issue Type: Bug Components: Index Affects Versions: 2.3.1 Reporter: Trejkaz Attachments: Test.java There is a situation where there is an IOException reading from Hits, and then the next time you get a NullPointerException instead of an IOException. Example stack traces: java.io.IOException: The specified network name is no longer available at java.io.RandomAccessFile.readBytes(Native Method) at java.io.RandomAccessFile.read(RandomAccessFile.java:322) at org.apache.lucene.store.FSIndexInput.readInternal(FSDirectory.java:536) at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:74) at org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:220) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:93) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:34) at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:57) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:88) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:344) at org.apache.lucene.index.IndexReader.document(IndexReader.java:368) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:84) at org.apache.lucene.search.Hits.doc(Hits.java:104) That error is fine. The problem is the next call to doc generates: java.lang.NullPointerException at org.apache.lucene.index.FieldsReader.getIndexType(FieldsReader.java:280) at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:216) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:101) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:344) at org.apache.lucene.index.IndexReader.document(IndexReader.java:368) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:84) at org.apache.lucene.search.Hits.doc(Hits.java:104) Presumably FieldsReader is caching partially-initialised data somewhere. I would normally expect the exact same IOException to be thrown for subsequent calls to the method. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
