Re: openJPA generates select per row - impossible to use for simple select statements

2009-07-21 Thread Kevin Sutter
JIRA Issue:  https://issues.apache.org/jira/browse/OPENJPA-1188

On Tue, Jul 21, 2009 at 8:42 AM, Kevin Sutter kwsut...@gmail.com wrote:

 For those of you interested, here are the details on this problem and
 what's required to reproduce it.  I will be opening a JIRA and posting a
 testcase shortly.

 Requirements...

 o  Your Entity needs to have an attribute that OpenJPA wrappers with a
 proxy.  This proxy is used to detect changes to these object types.  Example
 object types that OpenJPA wrappers are Calendar (culprit for this scenario),
 Date, Collection, and Map.

 o  In the Setter method for the proxied attribute, you must modify the
 value getting set.  In this scenario, the Calendar object was being modified
 in the setDate method via the set() method.

 o  The Entity must be using property-based access (annotations on the
 Getter methods).

 o  The Persistence Context must be clear of any Entities that are being
 queried.  For example, my testcase was persisting several Entities before
 executing the Query that was supposedly generating the extra SQL
 statements.  If I didn't clear the Persistence Context before executing the
 Query, the Entities were already populated and it didn't trigger the extra
 SQL statements.

 o  And, now comes the magic...  :-)  Access to the attribute's meta data
 seems to be done alphabetically.  From what I can tell, this seems to be a
 Java format convention.  In any case, the extra SQL statements are used to
 re-populate any attributes that are alphabetically after the proxied
 attribute that was modified in the corresponding Setter method.

 Given all of that setup, here's an explanation of what's happening...

 After the initial query is executed, the resulting Entity objects need to
 be populated with the data returned by the query.  When the setter was
 called for this proxied attribute, and the value was modified, this
 attribute (and the Entity) was marked as being dirty.  Part of the dirty
 processing is to determine which fields need to be re-loaded.  Of course,
 the proxied attribute does not have to be re-loaded since it was in the
 process of being modified.  The id field does not have to re-loaded since
 we're using id generation for this particular scenario.  And, any fields
 that are alphabetically before the proxied attribute were just loaded, so
 they don't have to be re-loaded.  But, any fields that come after the
 proxied attribute (alphabetically) don't realize that they will be loaded
 (due to the original query), so the extra SQL is pushed out to load these
 fields.

 As you can see, many stars had to align to reproduce this scenario.  I
 appreciate Mikhail's patience while we worked through the issue.  And, I
 appreciate the extra insights from Mike and Ravi.  This was actually quite
 fun to debug (in a sadistic point of view).  :-)

 Thanks,
 Kevin


 On Mon, Jul 20, 2009 at 10:40 AM, Kevin Sutter kwsut...@gmail.com wrote:

 I reproduced the problem!  There's something about the logic in Mikhail's
 resetTime method that is affecting the populating of the entities.  My
 resetTime() method had similar logic, but not exactly what Mikhail had.  I
 replaced the implementation and poof I reproduced the problem.

 I'll do some more experimenting to further isolate the problem and get a
 JIRA filed (maybe even resolved).

 Thanks, Mikhail, for your patience and assistance.  This was a strange
 one, indeed.

 Kevin


 On Mon, Jul 20, 2009 at 9:30 AM, om mikhail.ostrya...@ubs.com wrote:


 Hi,
 I don't have simple example. It looks like I can't help you here,
 unfortunately. This strange behavior occured on openJPA 1.0 embedded in
 WebSphere, OpenJPA 1.2.1 and last 2.0 versions linked to my application
 and
 initialized explicitly in code ( not through dependency injection ).
 Account
 object and persistence.xml are provided earlier, and they're extremely
 simple. Util.resetTime is extremely simple as well :
   public static Calendar resetTime(Calendar date) {
  if (date == null) return null;
  date.set(Calendar.HOUR_OF_DAY, 0);
  date.set(Calendar.MINUTE, 0);
  date.set(Calendar.SECOND, 0);
  date.set(Calendar.MILLISECOND, 0);
  return date;
   }

 WebSphere 6.1.0.21 with EJB3.0 pack, ORACLE version has been provided
 earlier (10.2.0.3 as I remember).
 Can't imagine what else can follow openJPA to this strange behavior...


 Kevin Sutter wrote:
 
  Hi Mikhail,
  If you can believe this, I still can not reproduce the problem.  What
  exactly does your util.resetTime() method do?
 
  I have been trying your scenario with old and new versions of OpenJPA.
  I
  even went back and used the same JVM (IBM JDK 5sr6b).  Still no luck.
 
  I can't any references in the spec or the OpenJPA documentation that
  restricts the code within getters and setters, but we do document how
 we
  proxy Date classes (and other similar types) so that we can more easily
  detect updates to these fields.  You have shown how this affects your
  usage.  

Re: FW: Memory leak

2009-07-21 Thread David Minor
Hi Mike,

I've tracked the problem down to
org.apache.openjpa.kernel.AbstractBrokerFactory. The heap dump lists the
_brokers Set as containing all the references to FinalizingBrokerImpl, and
it appears the assignment of this set was changed to this:

   if (FinalizingBrokerImpl.class.isAssignableFrom(
   bv.getTemplateBrokerType(_conf))) {
   return MapBackedSet.decorate(new ConcurrentHashMap(),
   new Object() { });
   } else {
   return new ConcurrentReferenceHashSet(
   ConcurrentReferenceHashSet.WEAK);
   }

It used to be assigned to the weak reference hash set as in the else
statement. Forcing the second assignment fixes the problem.


 -Original Message-
 From: Michael Dick [mailto:michael.d.d...@gmail.com]
 Sent: Tuesday, July 14, 2009 7:29 AM
 To: users@openjpa.apache.org
 Subject: Re: FW: Memory leak

 Hi David,

 There have been a few changes in PersistenceProviderImpl. One was to
 make the non-finalizing BrokerImpl the default (must be overridden in
 your
 config) another that might be interesting was adding a pool of
 EntityManagerFactories.

 From what I've seen the EMF pool is not used by default, but it's
 possible that the AbstractEntityManagerFactoryBean is setting it (the
 property is named  EntityManagerFactoryPool in case that helps). I took
 a quick pass at setting the pooling property and the only way I saw it
 take effect was to pass it in as a JVM arg (might be something in my
 eclipse env though - and I'm on 2.0.0-SNAPSHOT ATM).

 Hope this gives you a starting point, if not keep replying and we'll try
 to help

 -mike

 On Mon, Jul 13, 2009 at 8:00 PM, David Minor davemi...@gmail.com
 wrote:

 Hi Mike,

 Nothing else has changed. The application extends spring 2.0's
 AbstractEntityManagerFactoryBean class (apparently so that the
 persistence.xml file can be named something different).

 I notice it is checking the return type of
 AbstractEntityManagerFactoryBean's getPersistenceProvider() for an
 instance of openjpa's PersistenceProviderImpl, and doing something
 different depending on whether it finds it or not. Has anything
 changed with regards to this class?

  -Original Message-
  From: Michael Dick [mailto:michael.d.d...@gmail.com]
  Sent: Monday, July 13, 2009 12:49 PM
  To: users@openjpa.apache.org
  Subject: Re: Memory leak
 
  Hi David,
 
  FinalizingBrokerImpl will close itself and free resources when it's
  GC'ed.
  It sounds like something else is holding on to a lot of references
  to FBImpl (I'd guess something changed upstream).
 
  One cause is if the application creates a large number of
  EntityManagers and doesn't close them (or creates a large number of
  EMFactories which don't get closed since closing an EMF will close
 its EMs).
 
  Did anything else change or did you just upgrade OpenJPA versions?
 
  -mike
 
  On Mon, Jul 13, 2009 at 11:34 AM, David Minor davemi...@gmail.com
  wrote:
 
  Upgrading openjpa from 1.0.1 to 1.2.1 seems to introduce a memory
  leak
 
  in our application -- leaving the server running for a few days
  results in OOM errors (there are quartz tasks making simple openjpa

  selects during this time). A heap dump reveals
  org.apache.openjpa.kernel.FinalizingBrokerImpl as the dominant
  object,
 
  according to Eclipse's memory analysis plugin.
 
  Does anyone have an idea of what might be causing this?
 
  --
  _
  David Minor
 
 
 
 
 



 --
 _
 David Minor








-- 
_
David Minor


Newbie: sharing embedded Derby connection

2009-07-21 Thread Christopher Giblin

Hi,
I have  just completed the first  steps with OpenJPA running in my Eclipse
RCP app. I use Derby with the EmbeddedDriver. So far so good.

I have another set of plugins, Eclipse Data Tools Project (DTP), which
provide some DB management tools. However, both OpenJPA and DTP want to own
the Derby instance. When one has it, the other can not connect which is
actually to be expected.  When I close the EntityManagerFactory, it frees
the db. But of course, I want to create that factory just once.

I realize this is also a Derby question and equally a DTP question (I will
ask there, too).

Has anyone had to reconcile this before such that both could the embedded
Derby?

Thanks, chris