Reusing documents and invoking garbage collection slowed performance, but improved heap usage big time (From an average of 20 megs+ to about 12 megs+). I am wondering if there's anything else that can be done?
Grant Ingersoll-6 wrote: > > This question is better for java-user, but some questions for you: > > What version of Lucene are you using? The trunk now has the ability > to reuse Tokens/Documents, etc. to save on allocations that may help > reduce the amount of heap needed. The other thing to do is try > profiling to see where the allocations are coming from and then report > back to us. Also, info about your JVM, etc. would be helpful. > > -Grant > > > On Dec 27, 2007, at 9:01 AM, tgospodinov wrote: > >> >> I am indexing a collection of 100,000+ sentences in memory, as part >> of a >> client app. I tested the jvm heap consumption and it increases by >> about 40 >> megs. I tried indexing on disk (which I can't do in a production >> environment) just to test the heap usage, and I came up with about >> 20-25 >> megs. I have another search alternative, which is much simpler, but >> doesn't >> come close to Lucene, and it only consumes about 6-8 megs while >> indexing the >> same number of entries. Is there a way to improve heap usage while >> indexing >> in-memory? >> >> I appreciate your help. >> -- >> View this message in context: >> http://www.nabble.com/Improving-jvm-heap-consumption-when-indexing-tp14513435p14513435.html >> Sent from the Lucene - Java Developer mailing list archive at >> Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> > > -------------------------- > Grant Ingersoll > http://lucene.grantingersoll.com > > Lucene Helpful Hints: > http://wiki.apache.org/lucene-java/BasicsOfPerformance > http://wiki.apache.org/lucene-java/LuceneFAQ > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- View this message in context: http://www.nabble.com/Improving-jvm-heap-consumption-when-indexing-tp14513435p14514440.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
