Re: memory leak? - simple question
Thanks so much Kevin, that did the trick. Some of the issues you mentioned where introduced by me not copying right from the jUDDI project, but the main issue is that the openjpa-plugin simply does not run for some reason. Anyway it is no longer leaking now! I learned a lot from this exercise. And yes my class jumped from 2 to 10 k. BTW I'm pretty sure this used to work.. Thanks again! Going to fix it the jUDDI build now :)! --Kurt Kevin Sutter wrote: Hi Kurt, I have not run a profiler to verify the memory leak, but I think I found the problem... Although you thought you were running the PCEnhancer during build time, it was not being performed. When I looked at the Tomcat log with the original jar file, the following message was logged which indicates you were falling back to the not ready for production subclassing support: 1844 juddiDatabase INFO [http-8080-1] openjpa.Enhance - Creating subclass for [class org.apache.juddi.AuthToken]. This means that your application will be less efficient and will consume more memory than it would if you ran the OpenJPA enhancer. Additionally, lazy loading will not be available for one-to-one and many-to-one persistent attributes in types using field access; they will be loaded eagerly instead. I made a few changes to your pom.xml and the core module to get the enhancement to work and I believe the memory leak has went away. Like I said, I did not run a profiler, but I added a finalize method to your AuthToken class and I now see that these entities are getting GC'd. One thing I would do is add the following property to your persistence.xml. property name=openjpa.RuntimeUnenhancedClasses value=warn/ This property will produce the following warning if you accidentally fall into this subclassing support. We turn this off by default when running within WebSphere since we really don't want customers to accidentally use this subclassing. And, we have turned this off for 1.3.x and trunk due to the various problems associated with it. Unfortunately, the 1.2.x branch is kind of stuck with it... 1688 juddiDatabase WARN [http-8080-1] openjpa.Enhance - This configuration disallows runtime optimization, but the following listed types were not enhanced at build time or at class load time with a javaagent: [class org.apache.juddi.AuthToken]. And, of course, your webservice no longer works either. You will get an HTTP Status 500 error with an error message and call stack that further explains the error. To get the maven plugin to work properly... I made several changes, so hopefully I remember everything... o The number one thing is that the openjpa:enhance goal doesn't seem to get automatically invoked when you compile. I have no idea why, but that doesn't seem to work. o You have to make your persistence.xml file available to the core module. For my testing, I just created a new directory in the core module called \openjpa-leak\core\src\main\resources\META-INF\persistence.xml. This will automatically get copied over to target and then the openjpa:enhance goal will find it. o How you want to make your persistence.xml file available to both your core module and the leak-war module is up to you... :-) o I added a dependency in your core\pom.xml file for the 1.2.1 openjpa. The documentation [1] says it will default to 1.2.0, if none is specified. dependencies dependency groupIdorg.apache.openjpa/groupId artifactIdopenjpa/artifactId version1.2.1/version /dependency /dependencies o Your plugin properties don't seem to be specified correctly, at least according to the documentation, but I didn't touch them and things still seemed to work (once I got around the other problems). o I also changed where to find the entities to enhance since this pom.xml still specified the model directory. There also seems to be confusion as to whether to specify classes or includes. I went with includes since that's what the example showed. o It's very easy to see if the enhancement process worked or not since the size of your entity classes increases by a few thousand bytes. And, with the extra property above, this will prevent you from accidentally using the unenhanced version. Hope this helps! Let me know if this gets you around the memory leak. Kevin [1] http://mojo.codehaus.org/openjpa-maven-plugin/usage.html On Thu, Jan 14, 2010 at 2:12 PM, Kurt T Stam kurt.s...@gmail.com wrote: 1. Cool that you got it running :) 2. I just ran with openjpa 1.2.2.SNAPSHOT and still see an accumulation of AuthToken objects. --K Kevin Sutter wrote: Hey, this wasn't so difficult... :-) Maybe I can still learn new things... I have Tomcat up and running and I can get your app running. On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter kwsut...@gmail.com wrote: Hi Kurt, These instructions, of course, assume that I know
Re: memory leak? - simple question
Hi Kevin, I tried adding the javaagent stuff, and it still ends up leaking. I extracted the code into a tiny war with just one servlet which persist the authToken. The build creates a Tomcat distro backed by a derby database. You can download it from: http://people.apache.org/~kstam/leak.tgz You can build it with: mvn install, you can find the final tomcat in tomcat/target/tomcat-1.0.SNAPSHOT.zip or if you don't want to build it: http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip After you bring up tomcat it persistent an authToken when you hit http://localhost:8080/leak/test In the profiler I see an org.apache.juddi.AuthToken left behind for each request. Hopefully this little example will pinpoint what's going on. Thx Kev for sticking with me :) --Kurt Kevin Sutter wrote: Hi Kurt, Since you have the environment to reproduce this, could you run with the OpenJPA trace enabled? I'm not even sure what I'm looking for at this point. :-) But, maybe there's some tracing that would point us in the right direction. property name=openjpa.Log value=DefaultLevel=TRACE/ We have been always been on the watch for memory leaks, so this one is puzzling. You did mention blocked threads as well. At this point, you are probably hitting blocked threads due to the GC and Memory problems. But, we did recently resolve some locking situations [1] in our MetaDataRepository that could affect scaling (under extreme load). This was resolved in the 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so that should be coming soon). Does your WebService version of the problem use EntityManager transactions, or is the transaction service coming from Tomcat? And, does your WebService version also do the em.clear() and em.close()? Basically, are you using application-managed or container-managed persistence while running as a WebService? I'm very interested in helping to resolve this issue. Like I mentioned, we do a lot of memory leakage and scalability testing of the OpenJPA solution and we're not aware of anything. Although there is always the possibility that something was resolved in the 1.2.x branch after the 1.2.1 release that I'm not remembering... Thanks for your help, Kevin [1] https://issues.apache.org/jira/browse/OPENJPA-250 On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam kurt.s...@gmail.com wrote: Thanks Kevin, We're enhancing at build time: http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml Yeah we've been running load tests and things are nice and stable with Hibernate but with Openjpa we see increasing memory use, blocking threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our preference would be to ship with openjpa by default; but our build supports both hibernate and openjpa. And yes we use openjpa 1.2.1 (latest stable version). --Kurt Kevin Sutter wrote: Interesting detective work, Kurt. Thanks. Why the WebService version of the app would behave differently as far as GC is concerned is a mystery. And, you said that plugging in Hibernate into this scenario, everything works okay? Very confusing. How are you performing the Entity enhancement processing? Are you pre-enhancing via your build process? Or, are you using the -javaagent mechanism? Or, are you falling back to the subclassing support within OpenJPA? (See [1] for more information on these questions in case they don't make sense.) This would be one area that is different between Hibernate and OpenJPA -- enhancement processing. In the Tomcat environment, you may be falling back to the subclassing support (which we do not recommend) and hitting a memory leak with that. You said OpenJPA 1.2.x, right? Just a couple of thoughts on the subject... Kevin [1] http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam kurt.s...@gmail.com wrote: The same code executed straight from a java client (inVM) shows no memory leak. So is the fact that it is WebService significant then? What else can be different? I think one thread remains up, and somehow this causes openjpa not being able to clean up after itself. What can I do to debug this more? I can actually see in the profiler that the objects are allocated by the WebService, but why aren't they cleaned up? Thx, --Kurt Kurt T Stam wrote: Thanks Kevin, thanks for your response. I just replaced the static call by: apiAuthToken = new org.uddi.api_v3.AuthToken(); apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken()); //MappingModelToApi.mapAuthToken(modelAuthToken, apiAuthToken); which did not make a difference. I'm wondering if the fact that my class is a webservice makes a difference. I'll try extracting it into a regular class with a main method and profile that. At least I know that I didn't forget something completely obvious.. --Kurt Kevin Sutter
Re: memory leak? - simple question
Thanks Mark :). That looks much better. --Kurt Mark Struberg wrote: Btw (slightly OT) You could move addDefaultConstructor and enforcePropertyRestrictions in your pom.xml to the configuration section of the openjpa-maven-plugin without, having to use the toolProperties. This should do exactly the same, but is easier to read ;) configuration addDefaultConstructortrue/addDefaultConstructor enforcePropertyRestrictionstrue/enforcePropertyRestrictions /configuration LieGrue, strub --- On Thu, 1/14/10, Kurt T Stam kurt.s...@gmail.com wrote: From: Kurt T Stam kurt.s...@gmail.com Subject: Re: memory leak? - simple question To: users@openjpa.apache.org Date: Thursday, January 14, 2010, 4:27 AM Thanks Kevin, We're enhancing at build time: http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml Yeah we've been running load tests and things are nice and stable with Hibernate but with Openjpa we see increasing memory use, blocking threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our preference would be to ship with openjpa by default; but our build supports both hibernate and openjpa. And yes we use openjpa 1.2.1 (latest stable version). --Kurt Kevin Sutter wrote: Interesting detective work, Kurt. Thanks. Why the WebService version of the app would behave differently as far as GC is concerned is a mystery. And, you said that plugging in Hibernate into this scenario, everything works okay? Very confusing. How are you performing the Entity enhancement processing? Are you pre-enhancing via your build process? Or, are you using the -javaagent mechanism? Or, are you falling back to the subclassing support within OpenJPA? (See [1] for more information on these questions in case they don't make sense.) This would be one area that is different between Hibernate and OpenJPA -- enhancement processing. In the Tomcat environment, you may be falling back to the subclassing support (which we do not recommend) and hitting a memory leak with that. You said OpenJPA 1.2.x, right? Just a couple of thoughts on the subject... Kevin [1] http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam kurt.s...@gmail.com wrote: The same code executed straight from a java client (inVM) shows no memory leak. So is the fact that it is WebService significant then? What else can be different? I think one thread remains up, and somehow this causes openjpa not being able to clean up after itself. What can I do to debug this more? I can actually see in the profiler that the objects are allocated by the WebService, but why aren't they cleaned up? Thx, --Kurt Kurt T Stam wrote: Thanks Kevin, thanks for your response. I just replaced the static call by: apiAuthToken = new org.uddi.api_v3.AuthToken(); apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken()); //MappingModelToApi.mapAuthToken(modelAuthToken, apiAuthToken); which did not make a difference. I'm wondering if the fact that my class is a webservice makes a difference. I'll try extracting it into a regular class with a main method and profile that. At least I know that I didn't forget something completely obvious.. --Kurt Kevin Sutter wrote: Kurt, I agree that this is very common usage of the JPA programming model. And, we are not aware of any memory leaks. About the only thing that jumps out at me is the following two lines: apiAuthToken = new org.uddi.api_v3.AuthToken(); MappingModelToApi.mapAuthToken(modelAuthToken, apiAuthToken); What do these do? Can you comment these out and see if the memory leak still exists? Since you are passing the modelAuthToken into this method, I don't know what it's doing with the reference and could it be holding onto something to prevent the GC from cleaning up? The rest of your example seems very straight forward with creating and persisting objects. Kevin On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis curti...@gmail.com wrote: If you change the 1000 to something like 100... does your application go OOM? Are you running in a JSE environment? What is PersistenceManager? On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam kurt.s...@gmail.com wrote
Re: memory leak? - simple question
OK :). 1. Unzip tomcat-1.0.SNAPSHOT.zip 2. cd tomcat-1.0.SNAPSHOT/bin 3. ./start.sh Now, to send in a request open your browser to url: http://localhost:8080/leak/test Hit refresh to send in another request. I'll try getting you a trace log. Which packages? Just org.apache.openjpa? Cheers, --Kurt Kevin Sutter wrote: Hi Kurt, These instructions, of course, assume that I know something about Tomcat. This would not be a good assumption... :-) If you really want me to see this in action, I will need more information on how to start Tomcat and how to load applications. In the mean time, a trace log of OpenJPA would be an excellent start. Thanks! Kevin On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam kurt.s...@gmail.com wrote: Hi Kevin, I tried adding the javaagent stuff, and it still ends up leaking. I extracted the code into a tiny war with just one servlet which persist the authToken. The build creates a Tomcat distro backed by a derby database. You can download it from: http://people.apache.org/~kstam/leak.tgzhttp://people.apache.org/%7Ekstam/leak.tgz You can build it with: mvn install, you can find the final tomcat in tomcat/target/tomcat-1.0.SNAPSHOT.zip or if you don't want to build it: http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.ziphttp://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip After you bring up tomcat it persistent an authToken when you hit http://localhost:8080/leak/test In the profiler I see an org.apache.juddi.AuthToken left behind for each request. Hopefully this little example will pinpoint what's going on. Thx Kev for sticking with me :) --Kurt Kevin Sutter wrote: Hi Kurt, Since you have the environment to reproduce this, could you run with the OpenJPA trace enabled? I'm not even sure what I'm looking for at this point. :-) But, maybe there's some tracing that would point us in the right direction. property name=openjpa.Log value=DefaultLevel=TRACE/ We have been always been on the watch for memory leaks, so this one is puzzling. You did mention blocked threads as well. At this point, you are probably hitting blocked threads due to the GC and Memory problems. But, we did recently resolve some locking situations [1] in our MetaDataRepository that could affect scaling (under extreme load). This was resolved in the 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so that should be coming soon). Does your WebService version of the problem use EntityManager transactions, or is the transaction service coming from Tomcat? And, does your WebService version also do the em.clear() and em.close()? Basically, are you using application-managed or container-managed persistence while running as a WebService? I'm very interested in helping to resolve this issue. Like I mentioned, we do a lot of memory leakage and scalability testing of the OpenJPA solution and we're not aware of anything. Although there is always the possibility that something was resolved in the 1.2.x branch after the 1.2.1 release that I'm not remembering... Thanks for your help, Kevin [1] https://issues.apache.org/jira/browse/OPENJPA-250 On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam kurt.s...@gmail.com wrote: Thanks Kevin, We're enhancing at build time: http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml Yeah we've been running load tests and things are nice and stable with Hibernate but with Openjpa we see increasing memory use, blocking threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our preference would be to ship with openjpa by default; but our build supports both hibernate and openjpa. And yes we use openjpa 1.2.1 (latest stable version). --Kurt Kevin Sutter wrote: Interesting detective work, Kurt. Thanks. Why the WebService version of the app would behave differently as far as GC is concerned is a mystery. And, you said that plugging in Hibernate into this scenario, everything works okay? Very confusing. How are you performing the Entity enhancement processing? Are you pre-enhancing via your build process? Or, are you using the -javaagent mechanism? Or, are you falling back to the subclassing support within OpenJPA? (See [1] for more information on these questions in case they don't make sense.) This would be one area that is different between Hibernate and OpenJPA -- enhancement processing. In the Tomcat environment, you may be falling back to the subclassing support (which we do not recommend) and hitting a memory leak with that. You said OpenJPA 1.2.x, right? Just a couple of thoughts on the subject... Kevin [1] http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam kurt.s...@gmail.com wrote: The same code executed straight from a java client (inVM) shows no memory leak. So is the fact that it is WebService significant then? What else can be different? I think
Re: memory leak? - simple question
1. Cool that you got it running :) 2. I just ran with openjpa 1.2.2.SNAPSHOT and still see an accumulation of AuthToken objects. --K Kevin Sutter wrote: Hey, this wasn't so difficult... :-) Maybe I can still learn new things... I have Tomcat up and running and I can get your app running. On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter kwsut...@gmail.com wrote: Hi Kurt, These instructions, of course, assume that I know something about Tomcat. This would not be a good assumption... :-) If you really want me to see this in action, I will need more information on how to start Tomcat and how to load applications. In the mean time, a trace log of OpenJPA would be an excellent start. Thanks! Kevin On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam kurt.s...@gmail.com wrote: Hi Kevin, I tried adding the javaagent stuff, and it still ends up leaking. I extracted the code into a tiny war with just one servlet which persist the authToken. The build creates a Tomcat distro backed by a derby database. You can download it from: http://people.apache.org/~kstam/leak.tgzhttp://people.apache.org/%7Ekstam/leak.tgz You can build it with: mvn install, you can find the final tomcat in tomcat/target/tomcat-1.0.SNAPSHOT.zip or if you don't want to build it: http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.ziphttp://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip After you bring up tomcat it persistent an authToken when you hit http://localhost:8080/leak/test In the profiler I see an org.apache.juddi.AuthToken left behind for each request. Hopefully this little example will pinpoint what's going on. Thx Kev for sticking with me :) --Kurt Kevin Sutter wrote: Hi Kurt, Since you have the environment to reproduce this, could you run with the OpenJPA trace enabled? I'm not even sure what I'm looking for at this point. :-) But, maybe there's some tracing that would point us in the right direction. property name=openjpa.Log value=DefaultLevel=TRACE/ We have been always been on the watch for memory leaks, so this one is puzzling. You did mention blocked threads as well. At this point, you are probably hitting blocked threads due to the GC and Memory problems. But, we did recently resolve some locking situations [1] in our MetaDataRepository that could affect scaling (under extreme load). This was resolved in the 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so that should be coming soon). Does your WebService version of the problem use EntityManager transactions, or is the transaction service coming from Tomcat? And, does your WebService version also do the em.clear() and em.close()? Basically, are you using application-managed or container-managed persistence while running as a WebService? I'm very interested in helping to resolve this issue. Like I mentioned, we do a lot of memory leakage and scalability testing of the OpenJPA solution and we're not aware of anything. Although there is always the possibility that something was resolved in the 1.2.x branch after the 1.2.1 release that I'm not remembering... Thanks for your help, Kevin [1] https://issues.apache.org/jira/browse/OPENJPA-250 On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam kurt.s...@gmail.com wrote: Thanks Kevin, We're enhancing at build time: http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml Yeah we've been running load tests and things are nice and stable with Hibernate but with Openjpa we see increasing memory use, blocking threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our preference would be to ship with openjpa by default; but our build supports both hibernate and openjpa. And yes we use openjpa 1.2.1 (latest stable version). --Kurt Kevin Sutter wrote: Interesting detective work, Kurt. Thanks. Why the WebService version of the app would behave differently as far as GC is concerned is a mystery. And, you said that plugging in Hibernate into this scenario, everything works okay? Very confusing. How are you performing the Entity enhancement processing? Are you pre-enhancing via your build process? Or, are you using the -javaagent mechanism? Or, are you falling back to the subclassing support within OpenJPA? (See [1] for more information on these questions in case they don't make sense.) This would be one area that is different between Hibernate and OpenJPA -- enhancement processing. In the Tomcat environment, you may be falling back to the subclassing support (which we do not recommend) and hitting a memory leak with that. You said OpenJPA 1.2.x, right? Just a couple of thoughts on the subject... Kevin [1] http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam kurt.s...@gmail.com wrote: The same code executed straight from a java client (inVM) shows no memory leak. So is the fact
memory leak? - simple question
Hi guys,
[DESCRIPTION] The code below inserts a 1000 records in the database.
for (int i=1; i1000; i++) {
EntityManager em = PersistenceManager.getEntityManager();
EntityTransaction tx = em.getTransaction();
try {
tx.begin();
// Generate auth token and store it!
String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
org.apache.juddi.model.AuthToken modelAuthToken = new
org.apache.juddi.model.AuthToken();
if (authInfo != null) {
modelAuthToken.setAuthToken(authInfo);
modelAuthToken.setCreated(new Date());
modelAuthToken.setLastUsed(new Date());
modelAuthToken.setAuthorizedName(publisherId);
modelAuthToken.setNumberOfUses(0);
modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
em.persist(modelAuthToken);
}
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
tx.commit();
} finally {
if (tx.isActive()) {
tx.rollback();
}
em.clear();
em.close();
}
}
[ISSUE]
After it leaving this code I end up with a 1000
org.apache.juddi.model.AuthToken objects in memory. I've been using the
profiler, and these objects cannot be garbage collected.
This seems to be pretty the most common use case of using an OR-mapping
tool, so I find it hard to believe openjpa has a memory leak here. Does
anyone see what I'm doing wrong? Or can someone point me to an example
that does not exhibit this behavior? BTW same code using hibernate does
not accumulate these objects.
We're using openjpa 1.2.1.
Thx,
Kurt
Apache jUDDI.
Re: memory leak? - simple question
BTW I'm running with the cache off
property name=openjpa.DataCache value=false/
(that turns it off right?)
--Kurt
Kurt T Stam wrote:
Hi guys,
[DESCRIPTION] The code below inserts a 1000 records in the database.
for (int i=1; i1000; i++) {
EntityManager em = PersistenceManager.getEntityManager();
EntityTransaction tx = em.getTransaction();
try {
tx.begin();
// Generate auth token and store it!
String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
org.apache.juddi.model.AuthToken modelAuthToken = new
org.apache.juddi.model.AuthToken();
if (authInfo != null) {
modelAuthToken.setAuthToken(authInfo);
modelAuthToken.setCreated(new Date());
modelAuthToken.setLastUsed(new Date());
modelAuthToken.setAuthorizedName(publisherId);
modelAuthToken.setNumberOfUses(0);
modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
em.persist(modelAuthToken);
}
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
tx.commit();
} finally {
if (tx.isActive()) {
tx.rollback();
}
em.clear();
em.close();
}
}
[ISSUE]
After it leaving this code I end up with a 1000
org.apache.juddi.model.AuthToken objects in memory. I've been using
the profiler, and these objects cannot be garbage collected.
This seems to be pretty the most common use case of using an
OR-mapping tool, so I find it hard to believe openjpa has a memory
leak here. Does anyone see what I'm doing wrong? Or can someone point
me to an example that does not exhibit this behavior? BTW same code
using hibernate does not accumulate these objects.
We're using openjpa 1.2.1.
Thx,
Kurt
Apache jUDDI.
Re: memory leak? - simple question
Thanks Kevin, thanks for your response.
I just replaced the static call by:
apiAuthToken = new org.uddi.api_v3.AuthToken();
apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
//MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
which did not make a difference.
I'm wondering if the fact that my class is a webservice makes a
difference. I'll try extracting it into
a regular class with a main method and profile that. At least I know
that I didn't forget something
completely obvious..
--Kurt
Kevin Sutter wrote:
Kurt,
I agree that this is very common usage of the JPA programming model. And,
we are not aware of any memory leaks. About the only thing that jumps out
at me is the following two lines:
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
What do these do? Can you comment these out and see if the memory leak
still exists? Since you are passing the modelAuthToken into this method, I
don't know what it's doing with the reference and could it be holding onto
something to prevent the GC from cleaning up?
The rest of your example seems very straight forward with creating and
persisting objects.
Kevin
On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis curti...@gmail.com wrote:
If you change the 1000 to something like 100... does your application
go
OOM? Are you running in a JSE environment? What is PersistenceManager?
On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam kurt.s...@gmail.com wrote:
BTW I'm running with the cache off
property name=openjpa.DataCache value=false/
(that turns it off right?)
--Kurt
Kurt T Stam wrote:
Hi guys,
[DESCRIPTION] The code below inserts a 1000 records in the database.
for (int i=1; i1000; i++) {
EntityManager em = PersistenceManager.getEntityManager();
EntityTransaction tx = em.getTransaction();
try {
tx.begin();
// Generate auth token and store it!
String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
org.apache.juddi.model.AuthToken modelAuthToken = new
org.apache.juddi.model.AuthToken();
if (authInfo != null) {
modelAuthToken.setAuthToken(authInfo);
modelAuthToken.setCreated(new Date());
modelAuthToken.setLastUsed(new Date());
modelAuthToken.setAuthorizedName(publisherId);
modelAuthToken.setNumberOfUses(0);
modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
em.persist(modelAuthToken);
}
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
tx.commit();
} finally {
if (tx.isActive()) {
tx.rollback();
}
em.clear();
em.close();
}
}
[ISSUE]
After it leaving this code I end up with a 1000
org.apache.juddi.model.AuthToken objects in memory. I've been using the
profiler, and these objects cannot be garbage collected.
This seems to be pretty the most common use case of using an OR-mapping
tool, so I find it hard to believe openjpa has a memory leak here. Does
anyone see what I'm doing wrong? Or can someone point me to an example
that
does not exhibit this behavior? BTW same code using hibernate does not
accumulate these objects.
We're using openjpa 1.2.1.
Thx,
Kurt
Apache jUDDI.
--
Thanks,
Rick
Re: memory leak? - simple question
The same code executed straight from a java client (inVM) shows no
memory leak.
So is the fact that it is WebService significant then? What else can be
different? I think one thread remains up, and somehow this causes
openjpa not being able to clean up after itself. What can I do to debug
this more? I can actually see in the profiler that the objects are
allocated by the WebService, but why aren't they cleaned up?
Thx,
--Kurt
Kurt T Stam wrote:
Thanks Kevin, thanks for your response.
I just replaced the static call by:
apiAuthToken = new org.uddi.api_v3.AuthToken();
apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
//MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
which did not make a difference.
I'm wondering if the fact that my class is a webservice makes a
difference. I'll try extracting it into
a regular class with a main method and profile that. At least I know
that I didn't forget something
completely obvious..
--Kurt
Kevin Sutter wrote:
Kurt,
I agree that this is very common usage of the JPA programming model.
And,
we are not aware of any memory leaks. About the only thing that
jumps out
at me is the following two lines:
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
What do these do? Can you comment these out and see if the memory leak
still exists? Since you are passing the modelAuthToken into this
method, I
don't know what it's doing with the reference and could it be holding
onto
something to prevent the GC from cleaning up?
The rest of your example seems very straight forward with creating and
persisting objects.
Kevin
On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis curti...@gmail.com wrote:
If you change the 1000 to something like 100... does your
application
go
OOM? Are you running in a JSE environment? What is PersistenceManager?
On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam kurt.s...@gmail.com
wrote:
BTW I'm running with the cache off
property name=openjpa.DataCache value=false/
(that turns it off right?)
--Kurt
Kurt T Stam wrote:
Hi guys,
[DESCRIPTION] The code below inserts a 1000 records in the database.
for (int i=1; i1000; i++) {
EntityManager em = PersistenceManager.getEntityManager();
EntityTransaction tx = em.getTransaction();
try {
tx.begin();
// Generate auth token and store it!
String authInfo = AUTH_TOKEN_PREFIX +
UUID.randomUUID();
org.apache.juddi.model.AuthToken modelAuthToken = new
org.apache.juddi.model.AuthToken();
if (authInfo != null) {
modelAuthToken.setAuthToken(authInfo);
modelAuthToken.setCreated(new Date());
modelAuthToken.setLastUsed(new Date());
modelAuthToken.setAuthorizedName(publisherId);
modelAuthToken.setNumberOfUses(0);
modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
em.persist(modelAuthToken);
}
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
tx.commit();
} finally {
if (tx.isActive()) {
tx.rollback();
}
em.clear();
em.close();
}
}
[ISSUE]
After it leaving this code I end up with a 1000
org.apache.juddi.model.AuthToken objects in memory. I've been
using the
profiler, and these objects cannot be garbage collected.
This seems to be pretty the most common use case of using an
OR-mapping
tool, so I find it hard to believe openjpa has a memory leak here.
Does
anyone see what I'm doing wrong? Or can someone point me to an
example
that
does not exhibit this behavior? BTW same code using hibernate does
not
accumulate these objects.
We're using openjpa 1.2.1.
Thx,
Kurt
Apache jUDDI.
--
Thanks,
Rick
Re: memory leak? - simple question
Thanks Kevin,
We're enhancing at build time:
http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
Yeah we've been running load tests and things are nice and stable with
Hibernate but with Openjpa we see increasing memory use, blocking
threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267.
Our preference would be to ship with openjpa by default; but our build
supports both hibernate and openjpa.
And yes we use openjpa 1.2.1 (latest stable version).
--Kurt
Kevin Sutter wrote:
Interesting detective work, Kurt. Thanks.
Why the WebService version of the app would behave differently as far as GC
is concerned is a mystery. And, you said that plugging in Hibernate into
this scenario, everything works okay? Very confusing.
How are you performing the Entity enhancement processing? Are you
pre-enhancing via your build process? Or, are you using the -javaagent
mechanism? Or, are you falling back to the subclassing support within
OpenJPA? (See [1] for more information on these questions in case they
don't make sense.)
This would be one area that is different between Hibernate and OpenJPA --
enhancement processing.
In the Tomcat environment, you may be falling back to the subclassing
support (which we do not recommend) and hitting a memory leak with that.
You said OpenJPA 1.2.x, right?
Just a couple of thoughts on the subject...
Kevin
[1]
http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam kurt.s...@gmail.com wrote:
The same code executed straight from a java client (inVM) shows no memory
leak.
So is the fact that it is WebService significant then? What else can be
different? I think one thread remains up, and somehow this causes openjpa
not being able to clean up after itself. What can I do to debug this more? I
can actually see in the profiler that the objects are allocated by the
WebService, but why aren't they cleaned up?
Thx,
--Kurt
Kurt T Stam wrote:
Thanks Kevin, thanks for your response.
I just replaced the static call by:
apiAuthToken = new org.uddi.api_v3.AuthToken();
apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
//MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
which did not make a difference.
I'm wondering if the fact that my class is a webservice makes a
difference. I'll try extracting it into
a regular class with a main method and profile that. At least I know that
I didn't forget something
completely obvious..
--Kurt
Kevin Sutter wrote:
Kurt,
I agree that this is very common usage of the JPA programming model.
And,
we are not aware of any memory leaks. About the only thing that jumps
out
at me is the following two lines:
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
What do these do? Can you comment these out and see if the memory leak
still exists? Since you are passing the modelAuthToken into this method,
I
don't know what it's doing with the reference and could it be holding
onto
something to prevent the GC from cleaning up?
The rest of your example seems very straight forward with creating and
persisting objects.
Kevin
On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis curti...@gmail.com wrote:
If you change the 1000 to something like 100... does your
application
go
OOM? Are you running in a JSE environment? What is PersistenceManager?
On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam kurt.s...@gmail.com
wrote:
BTW I'm running with the cache off
property name=openjpa.DataCache value=false/
(that turns it off right?)
--Kurt
Kurt T Stam wrote:
Hi guys,
[DESCRIPTION] The code below inserts a 1000 records in the database.
for (int i=1; i1000; i++) {
EntityManager em = PersistenceManager.getEntityManager();
EntityTransaction tx = em.getTransaction();
try {
tx.begin();
// Generate auth token and store it!
String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
org.apache.juddi.model.AuthToken modelAuthToken = new
org.apache.juddi.model.AuthToken();
if (authInfo != null) {
modelAuthToken.setAuthToken(authInfo);
modelAuthToken.setCreated(new Date());
modelAuthToken.setLastUsed(new Date());
modelAuthToken.setAuthorizedName(publisherId);
modelAuthToken.setNumberOfUses(0);
modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
em.persist(modelAuthToken);
}
apiAuthToken = new org.uddi.api_v3.AuthToken();
MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);
tx.commit();
} finally {
if (tx.isActive
Re: Openjpa + MySQL + Tomcat datasource issue
Yup that was it :). We had a second file
(conf/Catalina/localhost/juddiv3.xml) with the datasource info. Going to
remove that file so it won't cause any more confusion!
Thx!
--Kurt
Miłosz Tylenda wrote:
Kurt,
Are you sure you are connecting to a MySQL database? The error message looks to
me more like coming from Derby than MySQL. I would say you are still connecting
to a Derby data source while having your OpenJPA properties set up for MySQL.
Cheers,
Milosz
Hi guys,
I'm working on the Apache jUDDI project and we use the JPA API for
persistence. One of the JPA implementations we use is OpenJPA. We've
been using Derby as our default database, which is working fine. However
when switching it to MySQL I'm getting the following error when OpenJPA
tries to create the juddiv3 tables:
SEVERE: StandardWrapper.Throwable
openjpa-1.2.1-r752877:753278 nonfatal general error
org.apache.openjpa.persistence.PersistenceException: Syntax error:
Encountered TYPE at line 1, column 163. {stmnt 7117250 CREATE TABLE
j3_address (id BIGINT NOT NULL, sort_code VARCHAR(10), tmodel_key
VARCHAR(255), use_type VARCHAR(255), address_id BIGINT NOT NULL, PRIMARY
KEY (id)) TYPE = InnoDB} [code=3, state=42X01]
at org.apache.openjpa.jdbc.meta.MappingTool.record(MappingTool.java:553)
at org.apache.openjpa.jdbc.meta.MappingTool.record(MappingTool.java:453)
at
org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory.synchronizeMappings(JDBCBrokerFactory.java:159)
at
org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory.newBrokerImpl(JDBCBrokerFactory.java:119)
at
org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:189)
at
org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:142)
at
org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:192)
at
org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:145)
at
org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:56)
at
org.apache.juddi.config.PersistenceManager.getEntityManager(PersistenceManager.java:45)
at
org.apache.juddi.config.AppConfig.getPersistentConfiguration(AppConfig.java:101)
The datasource (webapps/juddiv3/META-INF/context.xml) looks like:
Context
WatchedResourceWEB-INF/web.xml/WatchedResource
!--
Resource name=jdbc/JuddiDS auth=Container
type=javax.sql.DataSource username= password=
driverClassName=org.apache.derby.jdbc.EmbeddedDriver
url=jdbc:derby:juddi-derby-test-db;create=true
maxActive=8
/
--
!-- mysql --
Resource name=jdbc/JuddiDS auth=Container
type=javax.sql.DataSource username=root password=
driverClassName=com.mysql.jdbc.Driver
url=jdbc:mysql://localhost:3306/juddiv3
maxActive=8
/
/Context
The persistence.xml
(webapps/juddiv3/WEB-INF/classes/META-INF/persistence.xml) looks like
?xml version=1.0 encoding=UTF-8?
persistence xmlns=http://java.sun.com/xml/ns/persistence;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd;
version=1.0
persistence-unit name=juddiDatabase transaction-type=RESOURCE_LOCAL
providerorg.apache.openjpa.persistence.PersistenceProviderImpl/provider
non-jta-data-sourcejava:comp/env/jdbc/JuddiDS/non-jta-data-source
!-- entity classes --
classorg.apache.juddi.model.Address/class
classorg.apache.juddi.model.AddressLine/class
snip
classorg.apache.juddi.model.UddiEntityPublisher/class
properties
property name=openjpa.jdbc.SynchronizeMappings
value=buildSchema(SchemaAction='add')/
property name=openjpa.Log value=DefaultLevel=TRACE, Tool=INFO/
property name=openjpa.jdbc.UpdateManager value=operation-order/
property name=openjpa.jdbc.DBDictionary
value=org.apache.openjpa.jdbc.sql.MySQLDictionary /
/properties
/persistence-unit
Why would it not be liking the TYPE = InnoDB at the end of the create
statement (I want to use innodb)? The SQL looks fine to me. I'm using
MySQL5 and mysql-connector-java-5.1.6.jar. OpenJPA version 2.1.3, Tomcat
6.0.20.
Note that if reconfigure the persistence.xml to *not* use the tomcat
datasource, but rather connect to MySQL directly like so:
!-- mysql connection properties --
property name=openjpa.ConnectionURL
value=jdbc:mysql://localhost:3306/juddiv3/
property name=openjpa.ConnectionDriverName
value=com.mysql.jdbc.Driver/
property name=openjpa.ConnectionUserName value=root/
property name=openjpa.ConnectionPassword value=/
then things work fine!
Any help would be greatly appreciated! Does anyone have a working
example of how to use a
Is there any way to configure a table prefix across all entity tables?
Hi guys, I found this QA: Question: Is there any way to configure a table prefix across all entity tables? Answer: Not in the JPA specification, no. In OpenJPA, this can be accomplished by creating an extended DBDictionary and overriding the getValidTableName() method. and am wondering if anyone know if there is an option yet to do this through configuration, rather then code? Thx, --Kurt
