Re: Memory leak with ThreadGroups - and other stuff
Quick follow-on question for Craig... If you put a JDBC driver in your webapp's /WEB-INF/lib directory, then as that gets registered with DriverManager, what happens when you reload a context? If the DriverManager maintains a reference to the Driver loaded with the webapp classloader, that must surely cause a few problems for cleaning up the classloader... Should this sort of problem disappear with DriverManager.deregisterDriver() ? Are there other pitfalls of this sort in the standard Java APIs (I'm thinking of some classes with factory methods and helpful internal caching of instances created via such factory methods...) - Chris - Original Message - From: Craig R. McClanahan [EMAIL PROTECTED] Sent: Friday, January 24, 2003 1:49 AM Subject: RE: Memory leak with ThreadGroups If your application is well behaved (i.e. it doesn't have classes in common/lib or shared/lib that maintain references to things loaded from the webapp), then this will cause the entire webapp to become garbage. If *any* references to *any* classes inside the webapp still exist, though, then essentially nothing from your webapp can be collected. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Memory leak with ThreadGroups - and other stuff
-Original Message- From: Chris Brown [mailto:[EMAIL PROTECTED]] Sent: Friday, January 24, 2003 4:55 AM To: Tomcat Users List Subject: Re: Memory leak with ThreadGroups - and other stuff Quick follow-on question for Craig... If you put a JDBC driver in your webapp's /WEB-INF/lib directory, then as that gets registered with DriverManager, what happens when you reload a context? If the DriverManager maintains a reference to the Driver loaded with the webapp classloader, that must surely cause a few problems for cleaning up the classloader... When you reload the context, the webapp classloader and all its classes are discarded. Once a new classloader is created, it can not access the previous instance, so you will end up with ClassCastExceptions. Your old classloader will NOT be gc'ed until that reference is released. Should this sort of problem disappear with DriverManager.deregisterDriver() ? Are there other pitfalls of this sort in the standard Java APIs (I'm thinking of some classes with factory methods and helpful internal caching of instances created via such factory methods...) I have never used DriverManager.deregisterDriver(), but I would think that it would help. I have always had my database drivers in /common/lib so that they are only loaded once. if want you share your factory instances between webapps, put them in /common/lib. If you want each webapp to have its own instance of your factory methods and thus their own instances, then put them in WEB-INF/lib. The only other issue that I have run into with the web app classloader is when using System.loadLibrary() to load an external class. This causes a problem becuase the library is loaded by the JVM and the class instances are created by the classloader, so when you reload, you lose the instances and can't reload the library since the JVM already thinks its loaded. I got around this by leaving this code in /common/lib so it is only loaded once and shared among all webapps. Charlie - Chris - Original Message - From: Craig R. McClanahan [EMAIL PROTECTED] Sent: Friday, January 24, 2003 1:49 AM Subject: RE: Memory leak with ThreadGroups If your application is well behaved (i.e. it doesn't have classes in common/lib or shared/lib that maintain references to things loaded from the webapp), then this will cause the entire webapp to become garbage. If *any* references to *any* classes inside the webapp still exist, though, then essentially nothing from your webapp can be collected. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Memory leak with ThreadGroups
Thank-you for the reply. I have 4 java certifications but it seems like there has been little emphasis on class loaders which are actually very, very important. Could I ask another question? It seems like re-loading a new jsp workw easily and well, only the new jsp gets loaded, the application is not restarted. But, if a servelt is updated, Tomcat (if reload is on) begins a sequence: tries to serialize the session adn save it; reloads the app; then restore the session. Is it a class loader issue that prevents tomcat from treating the new servlet class tha same as a new jsp? -Original Message- From: Craig R. McClanahan [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 23, 2003 4:49 PM To: Tomcat Users List Subject: RE: Memory leak with ThreadGroups On Thu, 23 Jan 2003, Tolles, James wrote: Date: Thu, 23 Jan 2003 15:23:05 -0800 From: Tolles, James [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: 'Tomcat Users List' [EMAIL PROTECTED] Subject: RE: Memory leak with ThreadGroups I've noticed this when using jswat debugger. Each time we use the manager app to reload and updated class, we can see a new instace of the class in the jswat class list - each as a extra little number assigned to it. Its beyond my skill level but is seems odd that there can be two instances of the exact same class loaded in the VM? My guess is that the classloader loads in a new instance that is used but the previous one is still in the vm - not sure how this is possible. Classes have to be unique only within a class loader hierarchy, not JVM-wide. For instance, it's perfectly reasonable for two different webapps to have a class named com.mycompany.mypackage.Foo, even if those classes are totally different. Java does not provide any way to remove a previously loaded class from a class loader. The only way to implement webapp reloading, then, is to have Tomcat throw away its reference to the webapp class loader and create a new one. If your application is well behaved (i.e. it doesn't have classes in common/lib or shared/lib that maintain references to things loaded from the webapp), then this will cause the entire webapp to become garbage. If *any* references to *any* classes inside the webapp still exist, though, then essentially nothing from your webapp can be collected. Of course, this should only be an issue during development -- you should not use auto-reloading in a production environment. Craig -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Memory leak with ThreadGroups - and other stuff
On Fri, 24 Jan 2003, Chris Brown wrote: Date: Fri, 24 Jan 2003 10:54:38 +0100 From: Chris Brown [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Subject: Re: Memory leak with ThreadGroups - and other stuff Quick follow-on question for Craig... If you put a JDBC driver in your webapp's /WEB-INF/lib directory, then as that gets registered with DriverManager, what happens when you reload a context? If the DriverManager maintains a reference to the Driver loaded with the webapp classloader, that must surely cause a few problems for cleaning up the classloader... Probably. Should this sort of problem disappear with DriverManager.deregisterDriver() ? Are there other pitfalls of this sort in the standard Java APIs (I'm thinking of some classes with factory methods and helpful internal caching of instances created via such factory methods...) It might, but there's no way to know without seeing the source code to the driver itself. It might have other live references lying around. I tend to think of my JDBC driver as part of the system software, and put it in common/lib (so it doesn't get reloaded). This is required anyway, if you're going to use the JNDI data source support in Tomcat. - Chris Craig -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Memory leak with ThreadGroups
On Fri, 24 Jan 2003, Tolles, James wrote: It seems like re-loading a new jsp workw easily and well, only the new jsp gets loaded, the application is not restarted. But, if a servelt is updated, Tomcat (if reload is on) begins a sequence: tries to serialize the session adn save it; reloads the app; then restore the session. Is it a class loader issue that prevents tomcat from treating the new servlet class tha same as a new jsp? The JSP page compiler goes to an incredible amount of work to make that happen -- essentially, it creates a new class loader for every JSP page, and throws that class loader (and therefore the page class) away when you modify the page. This works for JSP because the Java code is all generated by the compiler (well, you can still screw up and cause memory leaks with your scriptlets, but there's nothing that any server can do about that), and it tries to ensure that only the container itself ever keeps a reference to the actual object for the generated page. Therefore, it can throw away its reference and have a high degree of confidence that the old object is now GC-able. For servlets (or for JavaBeans in general), there is no way for the container to know how many other objects have references to that servlet -- and that turns out to be pretty common, so the strategy would not be effective anyway. Some very early first-generation servlet containers were set up this way, but it turned out not to work in practice. Craig -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Memory leak with ThreadGroups
On Thu, 23 Jan 2003, Joseph Shraibman wrote: Date: Thu, 23 Jan 2003 16:58:28 -0500 From: Joseph Shraibman [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Memory leak with ThreadGroups One of my classes creates a static ThreadGroup. I notice that when I update a class and tomcat reloads that context the ThreadGroup is still around. If the ThreadGroup is still around that means the old class is still around, which means that class loader is still around. Why is it still around? I tried making the ThreadGroup daemon but it didn't help. I think the fact that the old class loaders are still around goes a long way to explaining why tomcat takes up so much memory. If your application creates new threads (and/or thread groups), then it's *your* problem to cleam them up when the application shuts down, not Tomcat's. Tomcat is only responsible for cleaning up its own threads. A very easy way to deal with this is create a ServletContextListener and clean up all your application threads in the contextDestroyed() method. Craig -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Memory leak with ThreadGroups
I've noticed this when using jswat debugger. Each time we use the manager app to reload and updated class, we can see a new instace of the class in the jswat class list - each as a extra little number assigned to it. Its beyond my skill level but is seems odd that there can be two instances of the exact same class loaded in the VM? My guess is that the classloader loads in a new instance that is used but the previous one is still in the vm - not sure how this is possible. -Original Message- From: Craig R. McClanahan [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 23, 2003 3:15 PM To: Tomcat Users List Subject: Re: Memory leak with ThreadGroups On Thu, 23 Jan 2003, Joseph Shraibman wrote: Date: Thu, 23 Jan 2003 16:58:28 -0500 From: Joseph Shraibman [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Memory leak with ThreadGroups One of my classes creates a static ThreadGroup. I notice that when I update a class and tomcat reloads that context the ThreadGroup is still around. If the ThreadGroup is still around that means the old class is still around, which means that class loader is still around. Why is it still around? I tried making the ThreadGroup daemon but it didn't help. I think the fact that the old class loaders are still around goes a long way to explaining why tomcat takes up so much memory. If your application creates new threads (and/or thread groups), then it's *your* problem to cleam them up when the application shuts down, not Tomcat's. Tomcat is only responsible for cleaning up its own threads. A very easy way to deal with this is create a ServletContextListener and clean up all your application threads in the contextDestroyed() method. Craig -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: Memory leak with ThreadGroups
On Thu, 23 Jan 2003, Tolles, James wrote: Date: Thu, 23 Jan 2003 15:23:05 -0800 From: Tolles, James [EMAIL PROTECTED] Reply-To: Tomcat Users List [EMAIL PROTECTED] To: 'Tomcat Users List' [EMAIL PROTECTED] Subject: RE: Memory leak with ThreadGroups I've noticed this when using jswat debugger. Each time we use the manager app to reload and updated class, we can see a new instace of the class in the jswat class list - each as a extra little number assigned to it. Its beyond my skill level but is seems odd that there can be two instances of the exact same class loaded in the VM? My guess is that the classloader loads in a new instance that is used but the previous one is still in the vm - not sure how this is possible. Classes have to be unique only within a class loader hierarchy, not JVM-wide. For instance, it's perfectly reasonable for two different webapps to have a class named com.mycompany.mypackage.Foo, even if those classes are totally different. Java does not provide any way to remove a previously loaded class from a class loader. The only way to implement webapp reloading, then, is to have Tomcat throw away its reference to the webapp class loader and create a new one. If your application is well behaved (i.e. it doesn't have classes in common/lib or shared/lib that maintain references to things loaded from the webapp), then this will cause the entire webapp to become garbage. If *any* references to *any* classes inside the webapp still exist, though, then essentially nothing from your webapp can be collected. Of course, this should only be an issue during development -- you should not use auto-reloading in a production environment. Craig -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Memory leak with ThreadGroups
One of my classes creates a static ThreadGroup. I notice that when I update a class and tomcat reloads that context the ThreadGroup is still around. If the ThreadGroup is still around that means the old class is still around, which means that class loader is still around. Why is it still around? I tried making the ThreadGroup daemon but it didn't help. I think the fact that the old class loaders are still around goes a long way to explaining why tomcat takes up so much memory. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: Memory leak with ThreadGroups
Sorry, forgot to mention: tomcat 4.1.18, jdk 1.4.1_01 on linux. Joseph Shraibman wrote: One of my classes creates a static ThreadGroup. I notice that when I update a class and tomcat reloads that context the ThreadGroup is still around. If the ThreadGroup is still around that means the old class is still around, which means that class loader is still around. Why is it still around? I tried making the ThreadGroup daemon but it didn't help. I think the fact that the old class loaders are still around goes a long way to explaining why tomcat takes up so much memory. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
