[ https://issues.apache.org/jira/browse/LUCENE-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001886#comment-13001886 ]
Hoss Man commented on LUCENE-2237: ---------------------------------- Note: this issue was created as an "improvement" and looks to add APIs to TimeLimitedCollector -- meanwhile LUCENE-2822 was created with the opinion that this is a bug and that TimeLimitedCollector is inherently broken. LUCENE-2822 has garnered some more discussion that should be considered by anyone interested in this issue. > allow TimeLimitedCollector timer thread to be shutdown > ------------------------------------------------------ > > Key: LUCENE-2237 > URL: https://issues.apache.org/jira/browse/LUCENE-2237 > Project: Lucene - Java > Issue Type: Improvement > Components: Search > Affects Versions: 2.4, 2.4.1, 2.9, 2.9.1, 3.0 > Reporter: Chris Darroch > Attachments: LUCENE-2237-2_4.patch > > > When using Tomcat 6 and Solr 1.3 (with Lucene 2.4) we found that if we caused > Tomcat to reload our .war files a number of times, we would eventually see > PermGen memory errors where the JVM' s GC reported that all "permanent > generation" memory had been consumed and none could be freed. This turns out > to be a fairly common issue when using Tomcat's autoDeploy feature (or > similar features of other application servers). See, for example: > http://ampedandwired.dreamhosters.com/2008/05/09/causes-of-java-permgen-memory-leaks/ > http://cornelcreanga.com/2009/02/how-to-prevent-memory-leaks-when-reloading-web-applications/ > http://www.samaxes.com/2007/10/classloader-leaks-and-permgen-space/ > http://blogs.sun.com/fkieviet/entry/how_to_fix_the_dreaded > My understanding of the issue is that when reloading a webapp, Tomcat starts > by releasing all of its references to the ClassLoader used to load the > previous version of the application. Then it creates a new ClassLoader which > reloads the application. The old ClassLoader and old version of the app are > left to the garbage collector to be cleaned up. However, if the app itself > hold references to the ClassLoader, the GC may not be able to ascertain that > the ClassLoader is truly unused, in which case, it and the entire old version > of app remain in memory. If one causes a sufficient number of app reloads, > eventually PermGen space is exhausted. > The particular issue we had with Solr and Lucene was that Lucene's > TimeLimitedCollector creates a thread which is not shut down anywhere; this > in turn seems to prevent Tomcat from unloading the ClassLoader. To solve > this I applied a minor patch to TimeLimitedCollector which adds a flag > variable controlling the timer thread's loop and some methods to set it so > the thread will exit. > The stopThread() method can then be called by an application such as Solr > from a class registered as a servlet context listener; when the server is > unloading the application the listener will execute and in turn stop the > timer thread. My testing during multiple reloads of solr.war with and > without these patches indicates that without them, we consistently get > PermGen errors, and with them, once the PermGen is nearly exhausted (which > may take a lot of reloads, e.g., 10-15!), the GC is able to free space and no > PermGen errors occur. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org