Re: index format error because disk full - possible Lucene bug?

2010-07-09 Thread Michael McCandless
I responded on the original thread. Disk full should never cause index corruption except on very old versions of Lucene... Mike On Thu, Jul 8, 2010 at 10:40 PM, Lance Norskog goks...@gmail.com wrote: I think he saying that there is a race condition involving writing out the buffered changes

Re: index format error because disk full - possible Lucene bug?

2010-07-09 Thread Earwin Burrfoot
I believe I've seen a similar condition a few times. A segments file referring zero-length segment files after a disk full event. On Fri, Jul 9, 2010 at 13:37, Michael McCandless luc...@mikemccandless.com wrote: I responded on the original thread. Disk full should never cause index corruption

Re: index format error because disk full - possible Lucene bug?

2010-07-09 Thread Michael McCandless
Which version of Lucene, and which OS/IO system? Is it possible fsync lies in your env? Mike On Fri, Jul 9, 2010 at 7:59 AM, Earwin Burrfoot ear...@gmail.com wrote: I believe I've seen a similar condition a few times. A segments file referring zero-length segment files after a disk full

Fwd: index format error because disk full - possible Lucene bug?

2010-07-08 Thread Lance Norskog
Forwarded to lucene-dev as a possible bug. On Wed, Jul 7, 2010 at 7:12 PM, Li Li fancye...@gmail.com wrote: I use SegmentInfos to read the segment_N file and found the error is that it try to load deletedDocs but the .del file's size is 0(because of disk error) . So I use SegmentInfos to set

Re: index format error because disk full - possible Lucene bug?

2010-07-08 Thread Lance Norskog
I think he saying that there is a race condition involving writing out the buffered changes from ram, doing deletes, doing commits and writing out new segment files when Lucene runs out of disk. This race condition (or possibly a swallowed I/O Exception) may cause bogus segment files, and the