Re: Corrupt Index with IndexWriter.addIndexes(IndexReader readers[])

2013-11-07 Thread Michael McCandless
OK, so CheckIndex found that the del files for 3 segments could not be found, e.g. it wanted to open _24xf_9l.del (yet it's _24xf_9k.del that's actually there). I wonder why CheckIndex doesn't report the exc you saw in flush, with that way-future segment (_33gg.cfs): that's weird. But ... I suspe

Re: Corrupt Index with IndexWriter.addIndexes(IndexReader readers[])

2013-11-07 Thread Gili Nachum
Thanks Mike and Uwe. I already reindexed in production, my goal is to get to the root cause to make sure it doesn't happen again. Will remove the flush(). No idea why it's there. Attaching checkIndex.Main() output (why did I bother writing my own output :#) *Output:* Opening index @ C:\\customers\

RE: Corrupt Index with IndexWriter.addIndexes(IndexReader readers[])

2013-11-06 Thread Uwe Schindler
Hi, > Hello, > I got an index corruption in production, and was wondering if it might be a > known bug (still with Lucene 3.1), or is my code doing something wrong. > It's a local disk index. No known machine power lose. No suppose to even > happen, right? > > This index that got corrupted is upda

Re: Corrupt Index with IndexWriter.addIndexes(IndexReader readers[])

2013-11-06 Thread Michael McCandless
No, it's not supposed to happen :) That exception is happening because IW is trying to apply the deletes during flush, and needs to open a reader for all existing segments to do so, and then finds that 33gg.cfs does not exist. Which is odd, because from your ls -l output, you have no segments eve

Re: Corrupt Index with IndexWriter.addIndexes(IndexReader readers[])

2013-11-06 Thread Gili Nachum
Following up with the structure of the index dir, and checkIndex output. *Structure of index:* 06/11/2013 10:55 AM . 06/11/2013 10:55 AM .. 02/11/2013 01:06 AM20 segments.gen 01/11/2013 03:00 AM 2,589 segments_29tx 02/11/2013 01:06 AM

Corrupt Index with IndexWriter.addIndexes(IndexReader readers[])

2013-11-05 Thread Gili Nachum
Hello, I got an index corruption in production, and was wondering if it might be a known bug (still with Lucene 3.1), or is my code doing something wrong. It's a local disk index. No known machine power lose. No suppose to even happen, right? This index that got corrupted is updated every 30sec; a