Hi Kurt, These instructions, of course, assume that I know something about Tomcat. This would not be a good assumption... :-) If you really want me to see this in action, I will need more information on how to start Tomcat and how to load applications. In the mean time, a trace log of OpenJPA would be an excellent start.
Thanks! Kevin On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <kurt.s...@gmail.com> wrote: > Hi Kevin, > > I tried adding the javaagent stuff, and it still ends up leaking. I > extracted the code into a tiny > war with just one servlet which persist the authToken. The build creates a > Tomcat distro backed by a > derby database. You can download it from: > > http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz> > > You can build it with: mvn install, you can find the final tomcat in > tomcat/target/tomcat-1.0.SNAPSHOT.zip > > or if you don't want to build it: > http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip> > > After you bring up tomcat it persistent an authToken when you hit > http://localhost:8080/leak/test > > In the profiler I see an org.apache.juddi.AuthToken left behind for each > request. > > Hopefully this little example will pinpoint what's going on. > > Thx Kev for sticking with me :) > > --Kurt > > > > Kevin Sutter wrote: > >> Hi Kurt, >> Since you have the environment to reproduce this, could you run with the >> OpenJPA trace enabled? I'm not even sure what I'm looking for at this >> point. :-) But, maybe there's some tracing that would point us in the >> right direction. >> >> <property name="openjpa.Log" value="DefaultLevel=TRACE"/> >> >> We have been always been on the watch for memory leaks, so this one is >> puzzling. You did mention blocked threads as well. At this point, you >> are >> probably hitting blocked threads due to the GC and Memory problems. But, >> we >> did recently resolve some locking situations [1] in our MetaDataRepository >> that could affect scaling (under extreme load). This was resolved in the >> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so >> that should be coming soon). >> >> Does your WebService version of the problem use EntityManager >> transactions, >> or is the transaction service coming from Tomcat? And, does your >> WebService >> version also do the em.clear() and em.close()? Basically, are you using >> application-managed or container-managed persistence while running as a >> WebService? >> >> I'm very interested in helping to resolve this issue. Like I mentioned, >> we >> do a lot of memory leakage and scalability testing of the OpenJPA solution >> and we're not aware of anything. Although there is always the possibility >> that something was resolved in the 1.2.x branch after the 1.2.1 release >> that >> I'm not remembering... >> >> Thanks for your help, >> Kevin >> >> [1] https://issues.apache.org/jira/browse/OPENJPA-250 >> >> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <kurt.s...@gmail.com> wrote: >> >> >> >>> Thanks Kevin, >>> >>> We're enhancing at build time: >>> >>> >>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml >>> >>> Yeah we've been running load tests and things are nice and stable with >>> Hibernate but with Openjpa we see increasing memory use, blocking threads >>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our >>> preference would be to ship with openjpa by default; but our build >>> supports >>> both hibernate and openjpa. >>> >>> And yes we use openjpa 1.2.1 (latest stable version). >>> >>> --Kurt >>> >>> >>> Kevin Sutter wrote: >>> >>> >>> >>>> Interesting detective work, Kurt. Thanks. >>>> >>>> Why the WebService version of the app would behave differently as far as >>>> GC >>>> is concerned is a mystery. And, you said that plugging in Hibernate >>>> into >>>> this scenario, everything works okay? Very confusing. >>>> >>>> How are you performing the Entity enhancement processing? Are you >>>> pre-enhancing via your build process? Or, are you using the -javaagent >>>> mechanism? Or, are you falling back to the subclassing support within >>>> OpenJPA? (See [1] for more information on these questions in case they >>>> don't make sense.) >>>> >>>> This would be one area that is different between Hibernate and OpenJPA >>>> -- >>>> enhancement processing. >>>> >>>> In the Tomcat environment, you may be falling back to the subclassing >>>> support (which we do not recommend) and hitting a memory leak with that. >>>> >>>> You said OpenJPA 1.2.x, right? >>>> >>>> Just a couple of thoughts on the subject... >>>> Kevin >>>> >>>> [1] >>>> >>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html >>>> >>>> >>>> >>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <kurt.s...@gmail.com> >>>> wrote: >>>> >>>> >>>> >>>> >>>> >>>>> The same code executed straight from a java client (inVM) shows no >>>>> memory >>>>> leak. >>>>> >>>>> So is the fact that it is WebService significant then? What else can be >>>>> different? I think one thread remains up, and somehow this causes >>>>> openjpa >>>>> not being able to clean up after itself. What can I do to debug this >>>>> more? I >>>>> can actually see in the profiler that the objects are allocated by the >>>>> WebService, but why aren't they cleaned up? >>>>> >>>>> Thx, >>>>> >>>>> >>>>> --Kurt >>>>> >>>>> >>>>> Kurt T Stam wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Thanks Kevin, thanks for your response. >>>>>> >>>>>> I just replaced the static call by: >>>>>> >>>>>> apiAuthToken = new org.uddi.api_v3.AuthToken(); >>>>>> apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken()); >>>>>> //MappingModelToApi.mapAuthToken(modelAuthToken, >>>>>> apiAuthToken); >>>>>> >>>>>> which did not make a difference. >>>>>> >>>>>> I'm wondering if the fact that my class is a webservice makes a >>>>>> difference. I'll try extracting it into >>>>>> a regular class with a main method and profile that. At least I know >>>>>> that >>>>>> I didn't forget something >>>>>> completely obvious.. >>>>>> >>>>>> --Kurt >>>>>> >>>>>> Kevin Sutter wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Kurt, >>>>>>> I agree that this is very common usage of the JPA programming model. >>>>>>> And, >>>>>>> we are not aware of any memory leaks. About the only thing that >>>>>>> jumps >>>>>>> out >>>>>>> at me is the following two lines: >>>>>>> >>>>>>> apiAuthToken = new org.uddi.api_v3.AuthToken(); >>>>>>> MappingModelToApi.mapAuthToken(modelAuthToken, >>>>>>> apiAuthToken); >>>>>>> >>>>>>> What do these do? Can you comment these out and see if the memory >>>>>>> leak >>>>>>> still exists? Since you are passing the modelAuthToken into this >>>>>>> method, >>>>>>> I >>>>>>> don't know what it's doing with the reference and could it be holding >>>>>>> onto >>>>>>> something to prevent the GC from cleaning up? >>>>>>> >>>>>>> The rest of your example seems very straight forward with creating >>>>>>> and >>>>>>> persisting objects. >>>>>>> >>>>>>> Kevin >>>>>>> >>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <curti...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> If you change the 1000 to something like 1000000... does your >>>>>>>> application >>>>>>>> go >>>>>>>> OOM? Are you running in a JSE environment? What is >>>>>>>> PersistenceManager? >>>>>>>> >>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <kurt.s...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> BTW I'm running with the cache off >>>>>>>>> >>>>>>>>> <property name="openjpa.DataCache" value="false"/> >>>>>>>>> >>>>>>>>> (that turns it off right?) >>>>>>>>> >>>>>>>>> --Kurt >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Kurt T Stam wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Hi guys, >>>>>>>>>> >>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the >>>>>>>>>> database. >>>>>>>>>> >>>>>>>>>> for (int i=1; i<1000; i++) { >>>>>>>>>> EntityManager em = PersistenceManager.getEntityManager(); >>>>>>>>>> EntityTransaction tx = em.getTransaction(); >>>>>>>>>> try { >>>>>>>>>> tx.begin(); >>>>>>>>>> // Generate auth token and store it! >>>>>>>>>> String authInfo = AUTH_TOKEN_PREFIX + >>>>>>>>>> UUID.randomUUID(); >>>>>>>>>> org.apache.juddi.model.AuthToken modelAuthToken = new >>>>>>>>>> org.apache.juddi.model.AuthToken(); >>>>>>>>>> if (authInfo != null) { >>>>>>>>>> modelAuthToken.setAuthToken(authInfo); >>>>>>>>>> modelAuthToken.setCreated(new Date()); >>>>>>>>>> modelAuthToken.setLastUsed(new Date()); >>>>>>>>>> modelAuthToken.setAuthorizedName(publisherId); >>>>>>>>>> modelAuthToken.setNumberOfUses(0); >>>>>>>>>> modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE); >>>>>>>>>> em.persist(modelAuthToken); >>>>>>>>>> } >>>>>>>>>> apiAuthToken = new org.uddi.api_v3.AuthToken(); >>>>>>>>>> MappingModelToApi.mapAuthToken(modelAuthToken, >>>>>>>>>> apiAuthToken); >>>>>>>>>> tx.commit(); >>>>>>>>>> } finally { >>>>>>>>>> if (tx.isActive()) { >>>>>>>>>> tx.rollback(); >>>>>>>>>> } >>>>>>>>>> em.clear(); >>>>>>>>>> em.close(); >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [ISSUE] >>>>>>>>>> After it leaving this code I end up with a 1000 >>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been >>>>>>>>>> using >>>>>>>>>> the >>>>>>>>>> profiler, and these objects cannot be garbage collected. >>>>>>>>>> >>>>>>>>>> This seems to be pretty the most common use case of using an >>>>>>>>>> OR-mapping >>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak here. >>>>>>>>>> Does >>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an >>>>>>>>>> example >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> that >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does >>>>>>>>> not >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> accumulate these objects. >>>>>>>>>> >>>>>>>>>> We're using openjpa 1.2.1. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thx, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Kurt >>>>>>>>>> >>>>>>>>>> Apache jUDDI. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> Thanks, >>>>>>>> Rick >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> >> >> >> > >
