It's great that CheckIndex resolved it, but I'd like to get to the root cause if possible.
Can you describe how the previous indexing process abnormally terminated? Did the JRE crash/get killed? Did the OS/machine crash / lose power? Abnormal termination of any kind, background merges running or not, should never cause this (unless we have a bug ;) ), except if fsync is not working correctly and the OS/machine had crashed. Mike On Mon, Jul 26, 2010 at 6:12 AM, David Sitsky <s...@nuix.com> wrote: > Running CheckIndex -fix fixed both indexes which is a relief. In both > cases, it said 1 broken segment containing 1 document detected. > > So any ideas on what might have caused this in the first place? > > On 26 July 2010 16:58, David Sitsky <s...@nuix.com> wrote: >> As another data point, this happened on another index: >> >> java.io.IOException: background merge hit exception: _17a:C1003710 >> _18h:C66391 _1ct:C45384 _1wt:C104479 _1zo:C23469 _271:C58570 >> _2dk:C53760 _2h5:C109879 _2ik:C7713 _2ii:C121 _2ij:C243 _2il:C124 >> _2im:C122 _2in:C115 _2io:C138 _2ip:C131 _2iq:C116 _2ir:C141 _2is:C183 >> _2it:C147 _2iu:C129 _2iv:C143 _2iw:C111 _2ix:C123 _2iy:C448 _2iz:C157 >> _2j0:C286 _2j1:C282 _2j2:C191 _2j3:C233 _2j4:C214 _2j5:C153 _2j6:C126 >> _2j7:C166 _2j8:C411 _2j9:C122 _2ja:C94 _2jb:C2840 _2jc:C70 _2jd:C3 >> _2je:C878 _2jf:C114 _2jg:C87 _2jh:C61 _2ji:C1 _2jj:C1 _2jk:C5 into >> _2jl [optimize] [mergeDocStores] >> ... >> Caused by: java.io.FileNotFoundException: D:\[snip]\_2ji.fdx (The >> system cannot find the file specified) >> >> We are going to try using the "check index" tool within Luke to see if >> this can help. >> >> In both instances, the Java program which has a single index writer >> instance did abnormally terminate, and automatically restarted to >> continue indexing. Is it possible if the program abnormally >> terminated while a background merge was occurring that this happen? >> In our program, we explicitly control when data is committed. >> >> From what I can see, the program continued to add more documents to >> the index after the restart without issue, before doing the final >> optimise operation, which is where the above error comes from, like >> the trace I showed before. >> >> On 26 July 2010 14:39, David Sitsky <s...@nuix.com> wrote: >>> Correction - this was with Lucene 2.9.3. >>> >>> On 26 July 2010 14:21, David Sitsky <s...@nuix.com> wrote: >>>> Hi, >>>> >>>> A customer has been indexing a very large collection of documents that >>>> has been running over many days using 2.9.0. During the optimisation >>>> stage, the following error occurred, and now the index can no longer >>>> be opened due to the "missing file". I have been told this index is >>>> on a local RAID disk configuration on a Windows machine. Is there >>>> anything we can do to somehow "recover" this index to make it useable? >>>> Any help would be greatly appreciated. >>>> >>>> Is this possible this is a bug with 2.9.0? >>>> >>>> java.io.IOException: background merge hit exception: _8d:C110892 >>>> _9s:C29548 _b7:C30930 _co:C20063 _e1:C43786 _fj:C102309 _gv:C67123 >>>> _ia:C26329 _jr:C61859 _l5:C38116 _mj:C33069 _ny:C14417 _pe:C13849 >>>> _qs:C21354 _s7:C10683 _u5:C11953 _v1:C12250 _wg:C18405 _xv:C24611 >>>> _zg:C15294 _10p:C20039 _125:C24706 _15d:C17375 _176:C73888 _17r:C13939 >>>> _19c:C9959 _1bz:C66884 _1dg:C6827 _1ex:C33196 _1g8:C16677 _1hn:C23601 >>>> _1j2:C11191 _1kh:C9764 _1lw:C8585 _1nb:C24527 _1oq:C24600 _1q5:C23088 >>>> _1rk:C10578 _1tl:C60504 _1ue:C23005 _1vu:C4571 _1vt:C6 _1vv:C6 _1vw:C1 >>>> _1vx:C4 _1vy:C1 _1vz:C2 _1w0:C1 _1w1:C1 _1w2:C26 into _1w3 [optimize] >>>> [mergeDocStores] >>>> at >>>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2908) >>>> at >>>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2843) >>>> at >>>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2813) >>>> [snip] >>>> ... >>>> Caused by: java.io.FileNotFoundException: D:\[snip]\TextIndex\_1vw.fdt >>>> (The system cannot find the file specified) >>>> at java.io.RandomAccessFile.open(Native Method) >>>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212) >>>> at >>>> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.<init>(SimpleFSDirectory.java:78) >>>> at >>>> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(SimpleFSDirectory.java:108) >>>> at >>>> org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:65) >>>> at >>>> org.apache.lucene.index.FieldsReader.<init>(FieldsReader.java:104) >>>> at >>>> org.apache.lucene.index.SegmentReader$CoreReaders.openDocStores(SegmentReader.java:291) >>>> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:654) >>>> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:622) >>>> at >>>> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:698) >>>> at >>>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5062) >>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4675) >>>> at >>>> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235) >>>> at >>>> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291) >>>> >>>> -- >>>> Cheers, >>>> David >>>> >>> >>> >>> >>> -- >>> Cheers, >>> David >>> >>> Nuix Pty Ltd >>> Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699 >>> Web: http://www.nuix.com Fax: +61 2 9212 6902 >>> >> >> >> >> -- >> Cheers, >> David >> >> Nuix Pty Ltd >> Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699 >> Web: http://www.nuix.com Fax: +61 2 9212 6902 >> > > > > -- > Cheers, > David > > Nuix Pty Ltd > Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699 > Web: http://www.nuix.com Fax: +61 2 9212 6902 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
