Bugs item #576913, was opened at 2002-07-03 15:07 Message generated for change (Comment added) made by sflexus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=376685&aid=576913&group_id=22866
Category: JBossServer Group: v3.0 Rabbit Hole Status: Open Resolution: None Priority: 5 Submitted By: Georg Schmid (giorgio42) Assigned to: Scott M Stark (starksm) Summary: OutOfMemory after redeploys Initial Comment: JBoss3.0.1RC1 (zip file from sf), Solaris8, JDK 1.3.1_01 After deploying the same ejb jar file about 30 times I got an OutOfMemory exception from the JVM. The package contains about 30 EJBs (entity and SLSBs) and is 450 K in size. The exception happened during the start of the redeployment. Regression? Georg ---------------------------------------------------------------------- Comment By: Alexei Yudichev (sflexus) Date: 2004-10-21 17:55 Message: Logged In: YES user_id=345880 I've spent a lot of time profiling this issue. My results are here: http://sourceforge.net/tracker/? func=detail&aid=973162&group_id=22866&atid=376685. java.security.ProtectionDomain are not in UCL3's GC path, because they themselves are referenced (indirectly) only by the same UCL3 instance. Among libraries integrated in JBoss, the problems are in: 1) Jasper's JSPWriter 2) Commons-logging LogFactory cache 3) somewhere else Fixing (1) and (2) helped me with all of my apps, except one, which I still can't understand why is leaking. The only hard reference to UCL3 is from some JAXB class I am using, but it itself is loaded by this UCL3... See the thread i linked above. ---------------------------------------------------------------------- Comment By: Wonne Keysers (wonnekeysers) Date: 2004-10-21 17:42 Message: Logged In: YES user_id=505480 java.security.ProtectionDomain instances seem to be holding a hard reference to the UnifiedClassLoader3 via their classloader attribute. As a result, the UnifiedClassLoader nor the classes it loaded will ever be garbage collected. The cause of this seems to be the pdcache HashMap, in the java.security.SecureClassLoader, which is never cleared. ---------------------------------------------------------------------- Comment By: Wonne Keysers (wonnekeysers) Date: 2004-09-29 13:53 Message: Logged In: YES user_id=505480 Dom4J references are also 'hanging' when using dom4j jar files isolated in .ear (hot-)deployments... http://sourceforge.net/tracker/index.php? func=detail&aid=1035511&group_id=16035&atid=116035 ---------------------------------------------------------------------- Comment By: freddy33 (freddy33) Date: 2004-09-23 01:48 Message: Logged In: YES user_id=1126772 We just run a full analysis of this problem using JProfiler and it looks like the UnifiedLoaderRepository3 doesn't remove the entry from one of its HashMap (loaderToResourcesMap) of the UnifiedClassLoader3 that was created for the previous deployment. To be clear every time we redeploy, a new UCL and HierarchicalLoaderRepository$NoParentClassLoader instances are leaking (meaning +1 after gabage collection after a redeploy). The UCL instance is not garbage collected because it appears as a key of the loaderToResourcesMap HashMap of the UnifiedLoaderRepository3. This issue is critical for us since the amount of classes in our ear (more than 5000) makes the memory leak huge and we cannot redeploy more than 6 times. Hope it helps, ---------------------------------------------------------------------- Comment By: Wonne Keysers (wonnekeysers) Date: 2004-09-22 16:02 Message: Logged In: YES user_id=505480 This message was posted on the log4j-user mailing list and might also be usefull in solving this bug? <http://www.mail-archive.com/log4j-user% 40logging.apache.org/msg01908.html> ---------------------------------------------------------------------- Comment By: Alexei Yudichev (sflexus) Date: 2004-09-09 11:08 Message: Logged In: YES user_id=345880 I have a question. If a redeploy an application, are its classes supposed to be reloaded and all objects referenced by static fields become subject to GC? I made a simple test by adding this code to one of my session bean classes: private static class GCChecker { public byte[] buf; public GCChecker() { buf = new byte[10 * 1024 * 1024]; System.out.println("[GCChecker] ALLOCATED 10Mb"); } protected void finalize() throws Throwable { System.out.println("[GCChecker] FINALIZED"); } } private final static GCChecker gcChecker = new GCChecker(); After 4 cycles of "bean usage attempt/app redeployment" I got OutOfMemoryException and in console there are only [GCChecker] ALLOCATED 10Mb messages. Is this a correct behaviour of class loaders and therefore I should set all static fields of all my classes to null at some application finalization stage? But then how about the class itself, how can I remove reference from classloader to loaded class? ---------------------------------------------------------------------- Comment By: Alexei Yudichev (sflexus) Date: 2004-09-07 18:52 Message: Logged In: YES user_id=345880 This is how I reproduce it on my Windows system, step by step: 1) Download JBoss 3.2.5 sources, build a distribution (SRC_HOME/build/build.bat) and assume JBOSS_HOME=SRC_HOME/build/output/jboss-3.2.5 2) Execute build.bat build testsuite module (testsuite/build. bat), copy SRC_HOME/testsuite/output/lib/jbosstest-web.ear to JBOSS_HOME/server/all/deploy 3) change JBOSS_HOME/bin/run.bat to specify -Xmx40m in JVM parameters (just no to wait long). I also included -Xloggc: c:/temp/jboss-gc.log to monitor memory 4) start JBoss 5) execute the following ant script, changing jboss.home accordingly: <?xml version="1.0" encoding="ISO-8859-1"?> <project name="JBossOOMGenerator" default="redeploy- endlessly"> <property name="jboss.home" value="C:/temp/4/jboss-3.2. 5"/> <property name="deployment.url" value="${jboss.home}/ server/all/deploy/jbosstest-web.ear"/> <path id="jmx.task.classpath"> <pathelement location="${jboss.home}/client/jbossall- client.jar"/> <pathelement location="${jboss.home}/docs/examples/jmx/ jbossjmx-ant.jar"/> </path> <target name="redeploy-endlessly"> <taskdef name="jmx" classname="org.jboss.ant.JMX" classpathref="jmx.task.classpath"/> <antcall target="loop"/> </target> <target name="loop"> <jmx adapterName="jmx/rmi/RMIAdaptor"> <propertyeditor type="java.net.URL" editor="org.jboss.util. propertyeditor.URLEditor"/> <invoke target="jboss.system:service=MainDeployer" operation="redeploy"> <parameter type="java.net.URL" arg="${deployment.url} "/> </invoke> </jmx> <antcall target="loop2"/> </target> <target name="loop2"> <antcall target="loop"/> </target> </project> In approx. 30 minutes (on my P4 2.4GHz) I get OutOfMemoryError in JBoss JVM. I believe if I set the -Xmx to around 35m I would get it much sooner. ---------------------------------------------------------------------- Comment By: IM (imaniuk) Date: 2003-12-03 22:13 Message: Logged In: YES user_id=923195 We are also experiencing this problem in 3.2.1 and 3.2.2. As a part of our solution we need to redeploy an ear file quite often and having to restart the server does not seem to be a nice fix. The memory leak occurs even if I remove all entries from application.xml. Nothing is being deployed, but the memory foot-print keeps growing... There seems to be two memory leaks: when runnig on Linux the top app reports SIZE and SHARE to grow with every redeploy. BTW, as a way to reproduce the problem I was redeploying web-console.war and that also resulted in momory usage growing, although at a much slower pace. ---------------------------------------------------------------------- Comment By: David Corbin (dcorbin) Date: 2003-10-30 17:09 Message: Logged In: YES user_id=1805 I have this too, with a simple .WAR file (not an EJB in site) :) I lose 30-40M of RAM per deployment cycle. (doing repeated deploy/undeploys, with no actual web requests) However, if I change our init code to do "nothing", I still lose memory, but more like 1-2M. I think that the entire webapp is still hanging around in memory, and never being subject to garbage collection, which leads me to believe somebody (in jboss) is holding on to a servlet reference. ---------------------------------------------------------------------- Comment By: David Corbin (dcorbin) Date: 2003-10-30 17:08 Message: Logged In: YES user_id=1805 I have this too, with a simple .WAR file (not an EJB in site) :) I lose 30-40M of RAM per deployment cycle. (doing repeated deploy/undeploys, with no actual web requests) However, if I change our init code to do "nothing", I still lose memory, but more like 1-2M. I think that the entire webapp is still hanging around in memory, and never being subject to garbage collection, which leads me to believe somebody (in jboss) is holding on to a servlet reference. ---------------------------------------------------------------------- Comment By: Rod Burgett (rodburgett) Date: 2003-08-21 22:58 Message: Logged In: YES user_id=681969 The number is correct, but it may be a bug # instead of patch. Try selecting bugs in the search category and then entering that number...it worked for me. ---------------------------------------------------------------------- Comment By: Tom D�hler (tom_doehler) Date: 2003-08-20 17:19 Message: Logged In: YES user_id=618539 Is this #757856 the right patch id, I can't find it ?? ---------------------------------------------------------------------- Comment By: Rod Burgett (rodburgett) Date: 2003-06-20 20:09 Message: Logged In: YES user_id=681969 I submitted a patch this morning, #757856, aimed at beefing up the cleanup method of DeploymentInfo. It wasn't intended to address this particular EAR deployment issue, but it might help. ---------------------------------------------------------------------- Comment By: Scott M Stark (starksm) Date: 2003-06-20 19:46 Message: Logged In: YES user_id=175228 I'm working on the issue and there is no current workaround. ---------------------------------------------------------------------- Comment By: Eric Jain (ejain) Date: 2003-06-20 19:18 Message: Logged In: YES user_id=474829 I can confirm this issue with JBoss 3.2.1 on Linux and Sun's Java 1.4.1. Loosing around 10MB of memory at every redeployment. The application consists of an ear with a nested war, jar and sar. Is anyone working on this issue, or is a workaround known (perhaps setting a scoped classloader)? ---------------------------------------------------------------------- Comment By: Tim McCune (javajedi) Date: 2003-03-28 21:22 Message: Logged In: YES user_id=62441 This is a duplicate of bug 435958, which is currently closed but I requested it be reopened. There is also some discussion of it in the forums, which used to be available at http://jboss.org/forums/thread.jsp?forum=121&thread=27888 before the forums all went to hell this week. I have had this problem with every version of JBoss I've tried, including 3.2RC4. We have seen the problem manifest itself on both Linux and Solaris, but interestingly, not on Windows. I talked to Scott Stark about this a couple days ago, and he mentioned that he runs a leak test, but that he normally runs it on Windows. I suggested he try it on Linux to at least recreate the problem. As for 30 redeploys not being a problem, our ear file is 30 MB, so we run out of memory after about 6 redeploys. Very much a pain for us right now. ---------------------------------------------------------------------- Comment By: David Ward (dward2) Date: 2003-03-27 16:03 Message: Logged In: YES user_id=526282 We've actually seen this in all JBoss versions we've used (2.4.x -> 3.x). Not that I don't want it fixed - I do - but we've come to the conclusion that we *never* do a hot-redploy "30 times" on a production server. In development maybe, but it's acceptable (at least for us) there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=376685&aid=576913&group_id=22866 ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ JBoss-Development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
