Hi Kevin, Try taking a heap dump snapshot and analyzing it with something like YourKit to see what's eating the memory. SPM for Solr (see signature) will show you JVM heap and GC numbers/graphs/activity that may shed some light on the issue. You could also turn on verbose GC logging and/or use jstat to understang GC activity some more.
Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Fri, Sep 21, 2012 at 4:50 PM, Kevin Goess <kgo...@bepress.com> wrote: > We're running Solr 3.4, a fairly out-of-the-box solr/jetty setup, with > -Xms1200m -Xmx3500m . When we start pushing more than a couple documents > per second at it (PDFs, they go through SolrCell/Tika/PDFBox), the java > process hangs, becoming completely unresponsive. > > We thought it might be an issue with PDFBox was problematic with lots of > blocked threads, but now we've also noticed that all the thread dumps show > a similar situation in the heap, where three of the areas are at 99%, even > though we don't get any out-of-memory messages. > > Heap > PSYoungGen total 796416K, used 386330K > eden space 398208K, 97% used > from space 398208K, 0% used > to space 398208K, 0% used > PSOldGen > object space 2389376K, 99% used > PSPermGen > object space 53824K, 99% used > > We've also just noticed that after restarting Solr, the PermGen space grows > steadily until it hits 99% and then just stays there. > > 1) Is that behavior of PermGen normal, growing steadily to 99% and then > staying there? Apparently we can increase PermSpace > to -XX:MaxPermSize=128M , but if there's a memory leak that only postpones > the problem. > > 2) If all three of those indicators are pegged at 99%, I would think that > the JVM would throw an out-of-memory exception, rather than just > withdrawing into its own navel, is that expected behavior or is it > indicative of anything else? > > Any tips would be greatly appreciated, thanks! > > -- > Kevin M. Goess > Software Engineer > Berkeley Electronic Press > kgo...@bepress.com > > 510-665-1200 x179 > www.bepress.com > > bepress: sustainable scholarly publishing
