Hey, this wasn't so difficult... :-) Maybe I can still learn new things... I have Tomcat up and running and I can get your app running.
On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kwsut...@gmail.com> wrote: > Hi Kurt, > These instructions, of course, assume that I know something about Tomcat. > This would not be a good assumption... :-) If you really want me to see > this in action, I will need more information on how to start Tomcat and how > to load applications. In the mean time, a trace log of OpenJPA would be an > excellent start. > > Thanks! > Kevin > > > On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <kurt.s...@gmail.com> wrote: > >> Hi Kevin, >> >> I tried adding the javaagent stuff, and it still ends up leaking. I >> extracted the code into a tiny >> war with just one servlet which persist the authToken. The build creates a >> Tomcat distro backed by a >> derby database. You can download it from: >> >> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz> >> >> You can build it with: mvn install, you can find the final tomcat in >> tomcat/target/tomcat-1.0.SNAPSHOT.zip >> >> or if you don't want to build it: >> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip> >> >> After you bring up tomcat it persistent an authToken when you hit >> http://localhost:8080/leak/test >> >> In the profiler I see an org.apache.juddi.AuthToken left behind for each >> request. >> >> Hopefully this little example will pinpoint what's going on. >> >> Thx Kev for sticking with me :) >> >> --Kurt >> >> >> >> Kevin Sutter wrote: >> >>> Hi Kurt, >>> Since you have the environment to reproduce this, could you run with the >>> OpenJPA trace enabled? I'm not even sure what I'm looking for at this >>> point. :-) But, maybe there's some tracing that would point us in the >>> right direction. >>> >>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/> >>> >>> We have been always been on the watch for memory leaks, so this one is >>> puzzling. You did mention blocked threads as well. At this point, you >>> are >>> probably hitting blocked threads due to the GC and Memory problems. But, >>> we >>> did recently resolve some locking situations [1] in our >>> MetaDataRepository >>> that could affect scaling (under extreme load). This was resolved in the >>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so >>> that should be coming soon). >>> >>> Does your WebService version of the problem use EntityManager >>> transactions, >>> or is the transaction service coming from Tomcat? And, does your >>> WebService >>> version also do the em.clear() and em.close()? Basically, are you using >>> application-managed or container-managed persistence while running as a >>> WebService? >>> >>> I'm very interested in helping to resolve this issue. Like I mentioned, >>> we >>> do a lot of memory leakage and scalability testing of the OpenJPA >>> solution >>> and we're not aware of anything. Although there is always the >>> possibility >>> that something was resolved in the 1.2.x branch after the 1.2.1 release >>> that >>> I'm not remembering... >>> >>> Thanks for your help, >>> Kevin >>> >>> [1] https://issues.apache.org/jira/browse/OPENJPA-250 >>> >>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <kurt.s...@gmail.com> >>> wrote: >>> >>> >>> >>>> Thanks Kevin, >>>> >>>> We're enhancing at build time: >>>> >>>> >>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml >>>> >>>> Yeah we've been running load tests and things are nice and stable with >>>> Hibernate but with Openjpa we see increasing memory use, blocking >>>> threads >>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our >>>> preference would be to ship with openjpa by default; but our build >>>> supports >>>> both hibernate and openjpa. >>>> >>>> And yes we use openjpa 1.2.1 (latest stable version). >>>> >>>> --Kurt >>>> >>>> >>>> Kevin Sutter wrote: >>>> >>>> >>>> >>>>> Interesting detective work, Kurt. Thanks. >>>>> >>>>> Why the WebService version of the app would behave differently as far >>>>> as >>>>> GC >>>>> is concerned is a mystery. And, you said that plugging in Hibernate >>>>> into >>>>> this scenario, everything works okay? Very confusing. >>>>> >>>>> How are you performing the Entity enhancement processing? Are you >>>>> pre-enhancing via your build process? Or, are you using the -javaagent >>>>> mechanism? Or, are you falling back to the subclassing support within >>>>> OpenJPA? (See [1] for more information on these questions in case they >>>>> don't make sense.) >>>>> >>>>> This would be one area that is different between Hibernate and OpenJPA >>>>> -- >>>>> enhancement processing. >>>>> >>>>> In the Tomcat environment, you may be falling back to the subclassing >>>>> support (which we do not recommend) and hitting a memory leak with >>>>> that. >>>>> >>>>> You said OpenJPA 1.2.x, right? >>>>> >>>>> Just a couple of thoughts on the subject... >>>>> Kevin >>>>> >>>>> [1] >>>>> >>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html >>>>> >>>>> >>>>> >>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <kurt.s...@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> The same code executed straight from a java client (inVM) shows no >>>>>> memory >>>>>> leak. >>>>>> >>>>>> So is the fact that it is WebService significant then? What else can >>>>>> be >>>>>> different? I think one thread remains up, and somehow this causes >>>>>> openjpa >>>>>> not being able to clean up after itself. What can I do to debug this >>>>>> more? I >>>>>> can actually see in the profiler that the objects are allocated by the >>>>>> WebService, but why aren't they cleaned up? >>>>>> >>>>>> Thx, >>>>>> >>>>>> >>>>>> --Kurt >>>>>> >>>>>> >>>>>> Kurt T Stam wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Thanks Kevin, thanks for your response. >>>>>>> >>>>>>> I just replaced the static call by: >>>>>>> >>>>>>> apiAuthToken = new org.uddi.api_v3.AuthToken(); >>>>>>> apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken()); >>>>>>> //MappingModelToApi.mapAuthToken(modelAuthToken, >>>>>>> apiAuthToken); >>>>>>> >>>>>>> which did not make a difference. >>>>>>> >>>>>>> I'm wondering if the fact that my class is a webservice makes a >>>>>>> difference. I'll try extracting it into >>>>>>> a regular class with a main method and profile that. At least I know >>>>>>> that >>>>>>> I didn't forget something >>>>>>> completely obvious.. >>>>>>> >>>>>>> --Kurt >>>>>>> >>>>>>> Kevin Sutter wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Kurt, >>>>>>>> I agree that this is very common usage of the JPA programming model. >>>>>>>> And, >>>>>>>> we are not aware of any memory leaks. About the only thing that >>>>>>>> jumps >>>>>>>> out >>>>>>>> at me is the following two lines: >>>>>>>> >>>>>>>> apiAuthToken = new org.uddi.api_v3.AuthToken(); >>>>>>>> MappingModelToApi.mapAuthToken(modelAuthToken, >>>>>>>> apiAuthToken); >>>>>>>> >>>>>>>> What do these do? Can you comment these out and see if the memory >>>>>>>> leak >>>>>>>> still exists? Since you are passing the modelAuthToken into this >>>>>>>> method, >>>>>>>> I >>>>>>>> don't know what it's doing with the reference and could it be >>>>>>>> holding >>>>>>>> onto >>>>>>>> something to prevent the GC from cleaning up? >>>>>>>> >>>>>>>> The rest of your example seems very straight forward with creating >>>>>>>> and >>>>>>>> persisting objects. >>>>>>>> >>>>>>>> Kevin >>>>>>>> >>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <curti...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> If you change the 1000 to something like 1000000... does your >>>>>>>>> application >>>>>>>>> go >>>>>>>>> OOM? Are you running in a JSE environment? What is >>>>>>>>> PersistenceManager? >>>>>>>>> >>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <kurt.s...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> BTW I'm running with the cache off >>>>>>>>>> >>>>>>>>>> <property name="openjpa.DataCache" value="false"/> >>>>>>>>>> >>>>>>>>>> (that turns it off right?) >>>>>>>>>> >>>>>>>>>> --Kurt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Kurt T Stam wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Hi guys, >>>>>>>>>>> >>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the >>>>>>>>>>> database. >>>>>>>>>>> >>>>>>>>>>> for (int i=1; i<1000; i++) { >>>>>>>>>>> EntityManager em = PersistenceManager.getEntityManager(); >>>>>>>>>>> EntityTransaction tx = em.getTransaction(); >>>>>>>>>>> try { >>>>>>>>>>> tx.begin(); >>>>>>>>>>> // Generate auth token and store it! >>>>>>>>>>> String authInfo = AUTH_TOKEN_PREFIX + >>>>>>>>>>> UUID.randomUUID(); >>>>>>>>>>> org.apache.juddi.model.AuthToken modelAuthToken = new >>>>>>>>>>> org.apache.juddi.model.AuthToken(); >>>>>>>>>>> if (authInfo != null) { >>>>>>>>>>> modelAuthToken.setAuthToken(authInfo); >>>>>>>>>>> modelAuthToken.setCreated(new Date()); >>>>>>>>>>> modelAuthToken.setLastUsed(new Date()); >>>>>>>>>>> modelAuthToken.setAuthorizedName(publisherId); >>>>>>>>>>> modelAuthToken.setNumberOfUses(0); >>>>>>>>>>> modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE); >>>>>>>>>>> em.persist(modelAuthToken); >>>>>>>>>>> } >>>>>>>>>>> apiAuthToken = new org.uddi.api_v3.AuthToken(); >>>>>>>>>>> MappingModelToApi.mapAuthToken(modelAuthToken, >>>>>>>>>>> apiAuthToken); >>>>>>>>>>> tx.commit(); >>>>>>>>>>> } finally { >>>>>>>>>>> if (tx.isActive()) { >>>>>>>>>>> tx.rollback(); >>>>>>>>>>> } >>>>>>>>>>> em.clear(); >>>>>>>>>>> em.close(); >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [ISSUE] >>>>>>>>>>> After it leaving this code I end up with a 1000 >>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been >>>>>>>>>>> using >>>>>>>>>>> the >>>>>>>>>>> profiler, and these objects cannot be garbage collected. >>>>>>>>>>> >>>>>>>>>>> This seems to be pretty the most common use case of using an >>>>>>>>>>> OR-mapping >>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak >>>>>>>>>>> here. >>>>>>>>>>> Does >>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an >>>>>>>>>>> example >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> that >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does >>>>>>>>>> not >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> accumulate these objects. >>>>>>>>>>> >>>>>>>>>>> We're using openjpa 1.2.1. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thx, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Kurt >>>>>>>>>>> >>>>>>>>>>> Apache jUDDI. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Thanks, >>>>>>>>> Rick >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >>> >> >> >
