Hi,
> Hello,
> I got an index corruption in production, and was wondering if it might be a
> known bug (still with Lucene 3.1), or is my code doing something wrong.
> It's a local disk index. No known machine power lose. No suppose to even
> happen, right?
>
> This index that got corrupted is updated every 30sec; adding to it a small
> delta's index (using addIndexes()) that was replicated from another machine.
> The series of writer actions to update the index is:
> 1. writer.deleteDocuments(q);
> 2. writer.flush(false, true);
> 3. writer.addIndexes(reader);
> 4. writer.commit(map);
>
> Is the index exposed to corruptions only during commit, or is addIndexes()
> risky by itself (doc says it's not).
> LUCENE-2610 <https://issues.apache.org/jira/browse/LUCENE-2610> kind of
> looks in the neberhood, though it's not a bug report.
Hi, LUCENE-2610 is completely unrelated, as this only affects
addIndexes(Directory...), not addIndexes(IndexReader...). The one you are using
is using the natural Lucene merging as it is done all the time while indexing
documents (Lucene internally uses the same code like addIndexes(IndexReader) to
merge segments). addIndexes(Directory) is very different and more risky to have
bugs in older Lucene versions (this one copies index files around without
touching them, but renaming them to have new segment names - which is somehow
"unnatural"; it also does not correctly lock the index directory in older
versions).
Why do you call flush() at all? I would leave that out, there is no reason to
do this from userland code.
To "repair" the index, use Checkindex command line tool with the "fix" option.
This will delete the segment that is missing (_33gg.cfs). Of course this data
is lost, but as the file is not there it is lost already. This will just remove
the metadata of this missing segment from your index. But before doing this,
you should check what checkindex prints out without fix option - the info you
posted is not the console output the tool prints when ran from command line and
run with assertions enabled (-ea JVM option). The output looks like the
"toString) of the Java API of CheckIndex class, which is not so helpful. Please
post the full output of the tool executed from command line:
java -cp lucene-core-3.1.0.jar org.apache.lucene.index.CheckIndex
<options....>
Uwe
> I'll add an ls -l output in a follow up email.
>
> Technically the first indication of problems is when calling flush, but it
> could
> be that the previous writer action left it broken for flush to fail.
> My stack trace is:
> Caused by: java.io.FileNotFoundException:
> /disks/data1/opt/WAS/LotusConnections/Data/catalog/index/Places/index/
> _33gg.cfs
> (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
> at
> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.
> <init>(SimpleFSDirectory.java:69)
> at
> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(Simp
> leFSDirectory.java:90)
> at
> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.<init>(NIOFSDire
> ctory.java:91)
> at
> org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
> at
> org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.
> java:66)
> at
> org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentRead
> er.java:113)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:578)
> at
> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:684)
> at
> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:659)
> at
> org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.jav
> a:283)
> at
> org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.jav
> a:191)
> at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3358)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3296)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
