It's failed on servers running SuSE 10.0 and 8.2 (ancient!) $ uname -a shows Linux phoebe 2.6.13-15-smp #1 SMP Tue Sep 13 14:56:15 UTC 2005 x86_64 x86_64 x86_64 GNU/Linux
and Linux phobos 2.4.20-64GB-SMP #1 SMP Mon Mar 17 17:56:03 UTC 2003 i686 unknown unknown GNU/Linux The first one has a 2.8Ghz Intel CPU, don't know about the second. I'll try and run the stress test. -- Ian. On Tue, Mar 18, 2008 at 2:17 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote: > > On Tue, Mar 18, 2008 at 7:38 AM, Ian Lea <[EMAIL PROTECTED]> wrote: > > Hi > > > > > > When bulk loading into a new index I'm seeing this exception > > > > Exception in thread "Thread-1" > > org.apache.lucene.index.MergePolicy$MergeException: > > org.apache.lucene.index.CorruptIndexException: doc counts differ for > > segment _4l: fieldsReader shows 67861 but segmentInfo shows 67862 > > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:271) > > Caused by: org.apache.lucene.index.CorruptIndexException: doc counts > > differ for segment _4l: fieldsReader shows 67861 but segmentInfo shows > > 67862 > > at > org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:313) > > at > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262) > > at > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221) > > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3093) > > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834) > > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240) > > > > when use java version 1.6.0_05-b13 or 1.6.0_04-b12 on linux, with > > lucene 2.3.0 or 2.3.1 or lucene-core-2.3-SNAPSHOT from yesterday. > > > > With java version 1.6.0_03-b05 things work fine. > > > > The exception happens a few hundred thousand documents into the load. > > > > A different program updating a different index with different data on > > a different server gave a similar error on version 1.6.0_05-b13 and > > lucene 2.3.0. > > > > Any ideas? Is this maybe a known issue or am I missing something obvious? > > My guess is perhaps a thread safety bug, more likely in Lucene > indexing code (less likely in the JVM or specific libc). > > What Linux version are you using? > What hardware are you running on (specifically, the CPU)? > > If possible, it would be great if you could check out Lucene trunk, > crank up the iterations by modifying the TestStressIndexing2 and maybe > fiddle with some of the other parameters in > TestStressIndexing2.testMultiConfig(), and see if you can get it to > fail. > > > -Yonik > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
