Ok, so my wife actually wrote a couple of month ago in Japanese about using strategy for leveraging the Insane library and a continuous integration server in order to prevent webapp classloader leakage issues from creeping in. If you can read Japanese, check out http://pdxwhitebox.blog118.fc2.com/blog-entry-33.html
For those of us who can't read Japanese, today she posted the English text it was based upon here: http://pdxwhitebox.blog118.fc2.com/blog-entry-161.html The information there is a little light on the specifics, but it does a great job at describing the strategy, the rationale, where to find Insane, and which API to use. When you read it, treat "plugins" as interchangeable with "webapps". Getting a little more specific, we leveraged Insane's ScannerUtils.scan() method to search for webapp classloaders that should not be reachable via a strong reference. Doing this involved (1) implementing a org.netbeans.insane.scanner.Visitor that gathers relevant classloaders and (2) implementing a org.netbeans.insane.scanner.Filter that causes weak/soft/phantom references to be skipped over by ScannerUtils.scan(). (1) To implement the Visitor class, just have visitObject(ObjectMap map, Object object) collect relevant classloaders passed as the second argument (probably instances of org.apache.catalina.loader.WebappClassLoader). All other methods can have empty implementations. (2) To implement the Filter class, have accept(Object obj, Object referredFrom, Field reference) return false if the second argument is an instance of SoftReference, WeakReference, or PhantomReference. Return true for everything else. With the Visitor and Filter implemented, the scanning code would look something like the following: Set<Object> roots = ScannerUtils.interestingRoots(); // add to this set anything else you'd like to make sure gets visits MyWebAppClassLoaderGatherer visitor = new MyWebAppClassLoaderGatherer(); MySkipWeakReferencesFilter filter = new MySkipWeakReferencesFilter(); ScannerUtils.scan( filter, visitor, roots, true ); // do whatever is needed with the classloaders gathered by the visitor Unfortunately, I don't remember why the last argument to ScannerUtils.scan() should be true, and I don't have the Insane source or javadoc handy to remind me. Sorry about that... With this in place, you can then setup your test environment to exercise a given webapp, shut it down, and then invoke your ScannerUtils code to see if that the webapp's classloader is still hanging around. This seems to work well with in combination with a continuous integration (CI) server whose job is to run tests against checkins all day. If your CI server tracks who checked it what, you'll be able to quickly narrow down what may have caused the leak, assuming that your web app wasn't leaking a classloader to begin with. A word of warning... this is a very heavy weight operation. If there are better ways this can be done, I'd love to hear about it! Best regards, Mark DeSpain -----Original Message----- From: Despain, Mark Sent: Tuesday, April 21, 2009 4:01 PM To: users@tomcat.apache.org Subject: RE: Headstart on "Resolving OOM-PermGen errors on webapp reload" Hi Chris, I'll follow up later tonight. Hopefully I'll have less typos then, but don't be surprised if I just go triple-negative instead :) ~Mark -----Original Message----- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Sent: Tuesday, April 21, 2009 3:47 PM To: Tomcat Users List Subject: Re: Headstart on "Resolving OOM-PermGen errors on webapp reload" -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark, On 4/21/2009 6:17 PM, mark_desp...@mcafee.com wrote: > None of the issues I've looked into have never been attributed to > Tomcat. You mean "ever" attributed to Tomcat, right? Good. ;) > * A webapp registering an object with another object that outlives > the webapp and forgetting to unregister it webapp shutdown. As a > result, that object cannot be garbage collected, which also prevents > that webapp's classloader from being garbage collected. It's worse than that: even if most of the objects created from classes loaded by that ClassLoader have gone out of scope and can be GC'd, the java.lang.Class, java.lang.Package, and java.lang.reflect.* objects that are created from those loaded classes won't be released until the ClassLoader is released. So if you have a single object that is being held by an object loaded by another ClassLoader, you can have potentially hundreds or thousands of objects left around in memory just wasting space. > * Instantiating a new Thread somehow (e.g. directly via its > constructor, or indirectly via a thread pool, a Timer, or a > ScheduledExecutorService) in response to a request to a webapp. This > is because, by default, the new Thread inherits the parent thread's > context classloader, which in Tomcat (5.5.x) is set to that webapp's > classloader. This is a very good point to consider, because many folks aren't aware of the thread->classloader relationship. If you don't kill your threads, you will keep your whole app's ClassLoader around forever. The 'daemonness' of the thread is not relevant. I'm looking at you, Quartz! > To help pro-actively detect webapp classloader garbage collection > issues, I've leveraged the Insane library (a library I found from > Netbeans while researching the topic) to write a utility that > searches for webapp classloaders that should have been garbage > collected. Using the utility in combination with automated tests has > been definitely helped catch and diagnose issues. Could you post some more info on this? I'm sure a lot of folks would be interested in this kind of thing. - -chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAknuTNYACgkQ9CaO5/Lv0PCN6wCgkDKwhzpRB7re4StuClVe1Rt/ 3K0Anj5eXjLiTql97dxbhrNFavPXGIvC =Iuos -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
