allow TimeLimitedCollector timer thread to be shutdown
------------------------------------------------------
Key: LUCENE-2237
URL: https://issues.apache.org/jira/browse/LUCENE-2237
Project: Lucene - Java
Issue Type: Improvement
Components: Search
Affects Versions: 3.0, 2.9.1, 2.9, 2.4.1, 2.4
Reporter: Chris Darroch
When using Tomcat 6 and Solr 1.3 (with Lucene 2.4) we found that if we caused
Tomcat to reload our .war files a number of times, we would eventually see
PermGen memory errors where the JVM' s GC reported that all "permanent
generation" memory had been consumed and none could be freed. This turns out
to be a fairly common issue when using Tomcat's autoDeploy feature (or similar
features of other application servers). See, for example:
http://ampedandwired.dreamhosters.com/2008/05/09/causes-of-java-permgen-memory-leaks/
http://cornelcreanga.com/2009/02/how-to-prevent-memory-leaks-when-reloading-web-applications/
http://www.samaxes.com/2007/10/classloader-leaks-and-permgen-space/
http://blogs.sun.com/fkieviet/entry/how_to_fix_the_dreaded
My understanding of the issue is that when reloading a webapp, Tomcat starts by
releasing all of its references to the ClassLoader used to load the previous
version of the application. Then it creates a new ClassLoader which reloads
the application. The old ClassLoader and old version of the app are left to
the garbage collector to be cleaned up. However, if the app itself hold
references to the ClassLoader, the GC may not be able to ascertain that the
ClassLoader is truly unused, in which case, it and the entire old version of
app remain in memory. If one causes a sufficient number of app reloads,
eventually PermGen space is exhausted.
The particular issue we had with Solr and Lucene was that Lucene's
TimeLimitedCollector creates a thread which is not shut down anywhere; this in
turn seems to prevent Tomcat from unloading the ClassLoader. To solve this I
applied a minor patch to TimeLimitedCollector which adds a flag variable
controlling the timer thread's loop and some methods to set it so the thread
will exit.
The stopThread() method can then be called by an application such as Solr from
a class registered as a servlet context listener; when the server is unloading
the application the listener will execute and in turn stop the timer thread.
My testing during multiple reloads of solr.war with and without these patches
indicates that without them, we consistently get PermGen errors, and with them,
once the PermGen is nearly exhausted (which may take a lot of reloads, e.g.,
10-15!), the GC is able to free space and no PermGen errors occur.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
