The schema generation tools might operate in unpredictable ways - potentially skewing the results.
Currently the test harness assumes that the schema tools will execute and be done executing by the time you've created the first EntityManager. This isn't guaranteed, some tools may not execute until the first time you access an entity. In that case your test is, unknowingly, skewed by the shema tools. I haven't looked at the tests to see whether this affects the OpenJPA results, I'll try to take a look later today if I can find some cycles. I do know that when SyncrhonizeMappings is found OpenJPA obtains a lot of connections. I put in a quick fix that reduced the use of connections, but I wouldn't say it's complete by any means. My recommendation would be to do the table generation separately. OpenJPA provides a standalone tool<http://openjpa.apache.org/builds/latest/docs/manual/manual.html#ref_guide_mapping_mappingtool>to generate tables, maybe the other providers have similar tools. That would save you the time of creating a separate persistence unit, at least. One other thing. It looks like the test expects each EntityManager to use a single connection until the EntityManager is closed. If that's the case you might want to add this property <property name="openjpa.ConnectionRetainMode" value="always"/> This setting is suitable when the application tightly controls the EntityManager, and wants to minimize connection pool use. It's not suitable in all scenarios (e.g. container managed EntityManagers), but if you want to force a single connection to be used this is the way to go. Hope this helps, -mike On Wed, Nov 10, 2010 at 8:16 PM, Ilan Kirsh <[email protected]> wrote: > > Hi Mike, > > Automatic schema generation is used by all the tested JPA implementations - > in order to simplify the benchmark. I agree, however, that it should be > excluded from time measurement because it is not a real factor in > production. I will try to explain what I did and I will be glad to get your > opinion. > > The time of every test run is divided into 2 phases. The first phase is > always excluded from time measurement. Only the performance in the second > phase is taken into account. This way, initialization operations including > class loading, enhancement by the Java Agent enhancer, JIT compilation, JVM > tuning (inlining and other optimizations) and schema generation - are all > excluded from time measurement. > > Connecting to the database (invocation of createEntityManager) is also > completed before starting to measure time to avoid the influence of > connection pools. > > Given the above, do you still think that the SynchronizeMappings property > might affect the results. If so, how? > > If necessary I may generate 2 persistence units for every test (per > ORM/DBMS) - one for initialization (including schema generation) and the > other for the time measurement. But I want to keep the benchmark simple as > possible, so I will do that only if it is really necessary. > > Best regards, > Ilan > -- > View this message in context: > http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5727348.html > Sent from the OpenJPA Users mailing list archive at Nabble.com. >
