I have to index 1650mb of documents, and eventually i will get "out of memory" with a RAMDir and get "too many open files" with a FSdir, so to get around this i am indexing 100 documents at a time in a RAMDir, then merge this RAMDir into a FSDir before i index the next set of 100 files. This made me work around both "out of memory" and "too many files" exceptions...
mvh karl řie -----Original Message----- From: Paul Friedman [mailto:[EMAIL PROTECTED]] Sent: 28. februar 2002 21:38 To: Lucene Users List Subject: Re: optimizing index - too many open files Sorry to bother y'all again. Found an answer in the archives under the Thread "Indexing problem". About to try using RAMDirectory first. pax et bonum. p. ----- Original Message ----- From: "Paul Friedman" <[EMAIL PROTECTED]> To: "Lucene Users List" <[EMAIL PROTECTED]> Sent: Thursday, February 28, 2002 1:23 PM Subject: optimizing index - too many open files > Hello all, > > I am running into an error: > java.io.FileNotFoundException: /lucene/index/_2vx.tii ( too many open > files ) > after my class calls IndexWriter.optimize(). > > Does anybody know what causes this error? > Any help is appreciated. > > ( By the way, the site that I am indexing is huge. > I have a crawler run through the site calling many .jsps, .pdfs, and .html > docs. > It ran fine two days ago after indexing 3700+ pages. ) > > Could the index be too large for Lucene to handle? > > The error: > java.io.FileNotFoundException: /lucene/index/-2vx.tii ( too many open > files ) > at java.io.RandomAccessFile.open( Native Method ) > at java.io.RandomAccessFile.<init> > at java.io.RandomAccessFile.<init> > at org.apache.lucene.store.FSInputStream$Descriptor.<init> > at org.apache.lucene.store.FSInputStream.<init> > at org.apache.lucene.store.FSDirectory.openFile > at org.apache.lucene.index.TermInfosReader.readIndex > at org.apache.lucene.index.TermInfosReader.<init> > at org.apache.lucene.index.SegmentReader.<init> > at org.apache.lucene.index.IndexWriter.mergeSegments > at org.apache.lucene.index.IndexWriter.optimize > > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > > -- > To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: <mailto:[EMAIL PROTECTED]> _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]> -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
