I don't have an explanation for this, but if it was me indexing this large amount of data I'd be running each of the 6 in a completely separate process. More control, less damage when one bit fails, perhaps better performance on a multi-processor machine. And perhaps you wouldn't get this problem!
-- Ian. [EMAIL PROTECTED] Chantal Ackermann wrote: > > hello all, > > I am still trying to find the best way to index a really big amount of data. > at the moment I am trying to index each of the 29 textfiles in a single > thread using for each an own IndexWriter and an own directory where to place > the index. there are always six threads working the same time. > > the problem that occures now is that every second thread stops due to a > FileNotFoundException or an ArrayIndexOutOfBoundsException (the latter only > once) while the other half finishes fine. the file's name is different for > each thread but has always the extension ".fnm". > > for example: > java.io.FileNotFoundException: > /lucenetest/medlineIndex/1976-1977/_2zfj.fnm (Datei oder > Verzeichnis nicht gefunden) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java(Compiled > Code)) at > java.io.RandomAccessFile.<init>(RandomAccessFile.java(Compiled Code)) > at org.apache.lucene.store.FSInputStream.<init>(Unknown Source) > at org.apache.lucene.store.FSInputStream.<init>(Unknown Source) > at org.apache.lucene.store.FSDirectory.openFile(Unknown Source) > at org.apache.lucene.index.FieldInfos.<init>(Unknown Source) > at org.apache.lucene.index.SegmentReader.<init>(Unknown Source) > at org.apache.lucene.index.IndexWriter.mergeSegments(Unknown Source) > at org.apache.lucene.index.IndexWriter.maybeMergeSegments(Unknown > Source) > at org.apache.lucene.index.IndexWriter.maybeMergeSegments(Unknown > Source) > at org.apache.lucene.index.IndexWriter.addDocument(Unknown Source) > at > >de.biomax.lucenetest.MedlineRecordIndexer.indexDocs(MedlineRecordIndexer.java(Compiled > Code)) > > since half of the files are indexed without throwing that kind of exception > I'm at a loss where to start debugging. any ideas? > > thanks a lot > chantal -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
