Re: Memory during Indexing

2009-03-12 Thread Michael McCandless
Niels Ott wrote: Hi Mark, markharw00d schrieb: Hi Niels, See the javadocs for IndexWriter.setRAMBufferSizeMB() I tried different settings. Apart from the fact that my memory issue seems to by my own fault, I'm wondering what Lucene does in the background. Apparently it does flush(),

Re: Memory during Indexing

2009-03-12 Thread Niels Ott
Michael McCandless schrieb: When RAM is full, IW flushes the pending changes to disk, but does not commit them, meaning external (newly opened or reopened) readers will not see the changes. Is there a built-in mechanism in the IndexReader to reload the index every now and then, after having

Re: Memory during Indexing

2009-03-12 Thread Grant Ingersoll
On Mar 12, 2009, at 10:47 AM, Niels Ott wrote: Michael McCandless schrieb: When RAM is full, IW flushes the pending changes to disk, but does not commit them, meaning external (newly opened or reopened) readers will not see the changes. Is there a built-in mechanism in the IndexReader to

Memory during Indexing

2009-03-11 Thread Niels Ott
Hi Lucene professionals! This may sound like a dumb beginner's question, but anyways: Can Lucene run out of memory during indexing? Should I use IndexWriter.flush() or .commit(), and if so, how often? Thank you for your support. Niels -- Niels Ott Computational Linguist (B.A.) http

Re: Memory during Indexing

2009-03-11 Thread markharw00d
Hi Niels, See the javadocs for IndexWriter.setRAMBufferSizeMB() Cheers Mark Niels Ott wrote: Hi Lucene professionals! This may sound like a dumb beginner's question, but anyways: Can Lucene run out of memory during indexing? Should I use IndexWriter.flush() or .commit(), and if so, how

Re: Memory during Indexing

2009-03-11 Thread Niels Ott
Hi Mark, markharw00d schrieb: Hi Niels, See the javadocs for IndexWriter.setRAMBufferSizeMB() I tried different settings. Apart from the fact that my memory issue seems to by my own fault, I'm wondering what Lucene does in the background. Apparently it does flush(), but not commit()? At

Out Of Memory during Indexing

2008-02-06 Thread Sirish Vadala
... ... ... ... ... Things work well, but not sure if there is any other better way to solve this problem. Thanks. Sirish -- View this message in context: http://www.nabble.com/Out-Of-Memory-during-Indexing-tp15312692p15312692.html Sent from the Lucene - Java Users mailing list archive at Nabble.com