RAMBufferSize id 50 Mb, i tried with 200 too the index is unoptimized MergeFactor is Default 10 and I have not changed it
MaxBuffered Docs is also default Saurabh Agarwal On Thu, May 27, 2010 at 10:31 PM, Erick Erickson <erickerick...@gmail.com>wrote: > What have you set various indexwriter properties to? Particularly > things like merge factor, max buffered docs and ram buffer size. > > The first thing I'd look at is MergeFactor. From the JavaDocs: > Determines how often segment indices are merged by addDocument(). With > smaller values, less RAM is used while indexing, and searches on > unoptimized > indices are faster, but indexing speed is slower. With larger values, more > RAM is used during indexing, and while searches on unoptimized indices are > slower, indexing is faster. Thus larger values (> 10) are best for batch > index creation, and smaller values (< 10) for indices that are > interactively > maintained. > > > HTH > Erick > > On Thu, May 27, 2010 at 11:52 AM, Saurabh Agarwal <srbh.g...@gmail.com > >wrote: > > > Hi, > > when I am running Lucene on a 512 MB system. > > I am getting the following error > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > > at > > > > > org.apache.lucene.index.DocumentsWriter$ByteBlockAllocator.getByteBlock(DocumentsWriter.java:1206) > > and sometimes > > An unexpected error has been detected by Java Runtime Environment: > > # Problematic frame: > > # J java.nio.ByteBuffer.arrayOffset()I > > > > > > if I decrease my corpus size it indexes it perfectly!!! > > > > Can someone tell me how to index a larger corpus even if it takes more > > time!!! > > > > Thanks > > > > Saurabh Agarwal > > >
