I will check it out!! Saurabh Agarwal
On Thu, May 27, 2010 at 11:13 PM, Erick Erickson <erickerick...@gmail.com>wrote: > The larger your RAMbufferSize, the more memory you consume FWIW. > > OK, then, does it always OOM on the same document? Are you trying to index > any particularly large documents? > > Erick > > On Thu, May 27, 2010 at 1:28 PM, Saurabh Agarwal <srbh.g...@gmail.com > >wrote: > > > RAMBufferSize id 50 Mb, i tried with 200 too > > the index is unoptimized > > MergeFactor is Default 10 and I have not changed it > > > > MaxBuffered Docs is also default > > Saurabh Agarwal > > > > > > On Thu, May 27, 2010 at 10:31 PM, Erick Erickson < > erickerick...@gmail.com > > >wrote: > > > > > What have you set various indexwriter properties to? Particularly > > > things like merge factor, max buffered docs and ram buffer size. > > > > > > The first thing I'd look at is MergeFactor. From the JavaDocs: > > > Determines how often segment indices are merged by addDocument(). With > > > smaller values, less RAM is used while indexing, and searches on > > > unoptimized > > > indices are faster, but indexing speed is slower. With larger values, > > more > > > RAM is used during indexing, and while searches on unoptimized indices > > are > > > slower, indexing is faster. Thus larger values (> 10) are best for > batch > > > index creation, and smaller values (< 10) for indices that are > > > interactively > > > maintained. > > > > > > > > > HTH > > > Erick > > > > > > On Thu, May 27, 2010 at 11:52 AM, Saurabh Agarwal <srbh.g...@gmail.com > > > >wrote: > > > > > > > Hi, > > > > when I am running Lucene on a 512 MB system. > > > > I am getting the following error > > > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap > space > > > > at > > > > > > > > > > > > > > org.apache.lucene.index.DocumentsWriter$ByteBlockAllocator.getByteBlock(DocumentsWriter.java:1206) > > > > and sometimes > > > > An unexpected error has been detected by Java Runtime Environment: > > > > # Problematic frame: > > > > # J java.nio.ByteBuffer.arrayOffset()I > > > > > > > > > > > > if I decrease my corpus size it indexes it perfectly!!! > > > > > > > > Can someone tell me how to index a larger corpus even if it takes > more > > > > time!!! > > > > > > > > Thanks > > > > > > > > Saurabh Agarwal > > > > > > > > > >
