Hi Shubbis,
On Wed, Mar 25, 2009 at 9:51 AM, Shubbis <marius.jo...@broadpark.no> wrote:
>
> Hi Michael,
>
> Sorry for the delay, but we have been very busy with the project and other
> mandatory school projects.
> Please don't mistake the delay for a lack of interest or enthusiasm.
>
> > Is the separate download the deciding factor or is it the experimentation
> > required to get the pool configured "right"? Including commons-dbcp in
> our
> > binary distribution is easy to fix. If it would help in bake-offs like
> this
> > I think we'd have no trouble getting the required votes for a change.
>
> Our project's guidelines specify that each framework has to be run with
> default functionality (or as close to as possible). For this specific
> project, with our guidelines, having dbcp included in the binary download
> would certainly allow it to be included.
>
> As you noted, this particular test doesn't benefit much from the pooling
> mechanism, but we are running quite a few other tests, so it would be
> interesting to include as much as possible.
>
I'll open a JIRA to include commons dbcp in the binary distribution. It
shouldn't increase the footprint too much.
As of right now though, it would seem that the procedure of opening new
> connections on every query was the culprit, and that ConnectionRetainMode
> has was the solution.
>
> I would like to ask though, if you could think of any reason a case like
> this would take much longer (5-10 sec) than say, EclipseLink to complete?
>
> Ex:
> Inserting a DeliveryCompany, with OneToOne relation to ContactInfo that has
> a ManyToOne to an Address that has an ManyToOne to an AreaCode that has a
> ManyToOne to a Country that has a ManyToOne to a Region.
>
> Now I realize that this is quite abstract, but I hope you get my point.
>
Nothing's jumping out at me. Might be an eager/lazy loading issue. Have you
specified any non-default loading properties in the entities, or are they
vanilla relationships?
>
> Shubbis
>
>
> Hi Shubbis,
>
> Glad to help. Some comments inline.
>
> On Sun, Mar 22, 2009 at 8:39 AM, Shubbis <marius.jo...@broadpark.no>
> wrote:
>
> >
> > Hi Michael and everyone else involved!
> >
> > I greatly appreciate the efforts put into this and I'm glad to report
> that
> > simple testing at home, with the same laptop I use during the project, it
> > seems that i get the desired performance when enabling
> ConnectionRetainMode
> > = always. I will of course continue testing when i get back to the
> office.
> > Just to clarify, I'm not sure this is the end of all my worries ;)
> >
> > Note. I am running this without connection pooling as this was not a
> > default
> > setting of OpenJPA. The project specifies that all frameworks are run
> with
> > minimal setting changes and "as vanilla as possible".
> >
>
>
>
> > This does however not explain why i get much better results on my home
> > laptop. Granted it is faster and has more memory, but I got the same
> > results
> > as Pinaki which would indicate (to me atleast) that the result on my home
> > laptop is as it's "supposed" to be.
>
>
> I'm not sure why you're getting different results. Your initial results
> roughly matched mine so I went with them as a baseline. Maybe my laptop
> and
> your work system are equally performance challenged :-).
>
> Now, a question. Since this ConnectionRetainMode is not on always by
> > default. Does this suggest that OpenJPA is supposed/(required?) to be run
> > with a connection pool addon? I mean, if this was the case, why is it not
> > included by default?
> >
>
> I wouldn't say a connection pool is required any more than a L2 cache is
> required. It really depends on your use case. As you can tell when you run
> with RetainMode and no pool, adding a pool doesn't always help and it can
> lead to other interesting problems. Paul mentioned some in an earlier
> email.
> In addition OpenJPA is often used with a JEE Application Server, which
> typically provides its own connection pool. Having two levels of pooling
> can
> lead to some interesting problems.
>
> I suspect these are some of the reasons why OpenJPA was originally
> distributed without an integrated connection pool (and you don't have to go
> far to get commons-dbcp so why reinvent the wheel).
>
> This specific test really benefits from having a single connection to the
> database. There's only one thread using the DB, and it's just doing reads.
> As long as the use case doesn't change you won't see much benefit from a
> connection pool (you only need one connection anyway), or a L2 cache (the
> EM
> has cached everything for you). Try setting the connection pool size to 1
> for EclipseLink and OpenJPA and you might be surprised at the results.
>
> When you start writing data and have multiple EMs involved then your
> options
> become more intersting. Then you'll see benefits with a connection pool /
> L2
> cache, etc.
>
> I hope this made any sense, I'm having a hard time formulating coherent
> > replies as this is my second language and I'm quite new to all of this
> > (OpenJPA/JPA etc..).
> >
>
> You're doing better than I would do in my second language, I'm sure.
>
> -mike
>
>
> > Thanks again!
> >
> > Shubbis
> >
> >
> >
> > Michael Dick wrote:
> > >
> > > Hi all,
> > >
> > > As Paul pointed out privately I didn't indicate which versions of
> OpenJPA
> > > I
> > > was using. I've been testing with 1.2.0, 1.2.1 and 2.0.0-SNAPSHOT
> > > primarily.
> > > I also ran with 1.3.0-SNAPSHOT, and 1.0.3 for comparison - there wasn't
> > > much
> > > difference though so reduced the scope to 1.2 and trunk.
> > >
> > > The testcase uses a single EntityManager instance to issue a batch of
> > > findBy
> > > operations. In OpenJPA each findBy (or any query for that matter)
> obtains
> > > a
> > > connection from the connection pool and returns it after getting the
> > > results. So we're spending a lot of time moving the connection to and
> > from
> > > the pool (some cleanup is done along the way).
> > >
> > > Fortunately this behavior can be configured with the
> > > openjpa.ConnectionRetainMode property. Setting it to "always" causes
> the
> > > EntityManager to hold on to a single connection until the EntityManager
> > > closes. Obviously this setting introduces the possibility of exhausting
> > > the
> > > connection pool if num_entitymanagers > max_connections, but for this
> > > benchmark it's safe to try.
> > >
> > > Setting ConnectionRetainMode gave OpenJPA equivalent times on my laptop
> > at
> > > 100 - 100,000 iterations. In addition I removed the
> > > openjpa.jdbc.SynchronizeMappings property from the example (it's
> > > extraneous
> > > once the tables are created anyway).Another option I enabled that made
> > > some
> > > difference was preparedStatementCaching in dbcp. I'm assuming
> EclipseLink
> > > has some pstmt caching as well, but that could be faulty - in which
> case
> > > I'll disable it in dbcp.
> > >
> > > Here's the entire set of properties I'm using :
> > > properties.put("openjpa.Log", "DefaultLevel=FATAL");
> > > properties.put("openjpa.RuntimeUnenhancedClasses",
> > "unsupported");
> > > properties.put("openjpa.ConnectionRetainMode", "always");
> > > properties.put("openjpa.ConnectionDriverName",
> > > "org.apache.commons.dbcp.BasicDataSource");
> > >
> > > properties.put("openjpa.ConnectionProperties", String.format(
> > > "DriverClassName=%s, Url=%s, username=%s, password=%s,"
> > > + " MaxActive=%s, MaxIdle=%s, MinIdle=%s, MaxWait=%s"
> > > + ", poolPreparedStatements=true"
> > > , JDBC_DRIVER, JDBC_URL, JDBC_USER,
> > > JDBC_PASSWORD, MAX_CON, MIN_CON, MIN_CON, "1000"));
> > >
> > > EntityManagerFactory factory =
> > > Persistence.createEntityManagerFactory("OpenJPAPU",
> > > properties);
> > >
> > > MIN_CON = 1, MAX_CON=10.
> > >
> > > Shubbis, could you try running something similar and see if you get the
> > > same
> > > results?
> > >
> > > -mike
> > >
> > > On Fri, Mar 20, 2009 at 8:46 AM, Michael Dick
> > > <michael.d.d...@gmail.com>wrote:
> > >
> > >> Hi, I took a quick run with the source code from the RAR Shubbis
> > attached
> > >> earlier (thanks BTW).
> > >>
> > >> The SQL we execute for this findBy is SELECT t0.warehouseName FROM
> > >> Warehouse t0 WHERE t0.warehouseNr = ?. Pretty basic, and I doubt
> > >> EclipseLink
> > >> is doing better (certainly not 3x) based solely on the SQL.
> > >>
> > >> So I started digging deeper and on my laptop (not to be confused with
> > any
> > >> sort of "real" benchmark) there's a sweet spot around 100 iterations.
> > >> Under
> > >> 100 OpenJPA is faster. Between 100 and 125 they're comparable, and
> over
> > >> 125
> > >> iterations EclipseLink starts pulling ahead.
> > >>
> > >> Environment :
> > >> * Entities were enhanced by the PCEnhancer tool prior to running the
> > >> tests.
> > >>
> > >> * Connection pooling is enabled for EclipseLink and OpenJPA with
> roughly
> > >> the same settings. EclipseLink's pool and commons-dbcp weren't an easy
> > >> 1:1
> > >> match, so I might have some investigation to do there.
> > >> * MySQL Connector for Java v 5.1.7.
> > >> * MySQL database running locally, Version: 5.0.67-0ubuntu6
> > >> * Tests executed in Eclipse, YMMV outside of Eclipse.
> > >> * Sun JDK 5 java full version "1.5.0_15-b04"
> > >>
> > >> I have done a lot of hacking about with the sample application but I
> > >> don't
> > >> think I've violated the intent of the exercise. I'll upload the app to
> a
> > >> jira shortly.
> > >>
> > >> The relevant code is in my pastebin at these links :
> > >> persistence.xml : http://miked.pastebin.com/m490814b7
> > >> test01.java : http://miked.pastebin.com/m7d3df62f
> > >> WarehouseDAO.java : http://miked.pastebin.com/m49ab9a0e
> > >>
> > >> I highlighted the changed lines in WarehouseDAO, but missed it on the
> > >> others (too many lines to highlight accurately.
> > >>
> > >> I'm still looking, but thought this was worth sharing in case someone
> > >> else
> > >> sees something I've missed.
> > >>
> > >> -mike
> > >>
> > >>
> > >>
> > >> On Thu, Mar 19, 2009 at 10:53 AM, Paul Copeland
> > >> <t...@jotobjects.com>wrote:
> > >>
> > >>>
> > >>> At one point in this thread it was mentioned that the benchmark ran
> > much
> > >>> faster on a home computer than on an office computer and the reason
> for
> > >>> the
> > >>> difference was not obvious. Was that difference explained yet?
> > >>>
> > >>> What version of OpenJPA is the test using?
> > >>>
> > >>> - Paul
> > >>>
> > >>>
> > >>> On 3/19/2009 7:44 AM, Kevin Sutter wrote:
> > >>>
> > >>>> Shubbis and Nitish,
> > >>>> Thanks for your efforts. So, to clarify -- all implementations are
> > >>>> using
> > >>>> similar configurations (ie. connection pooling, caching,
> enhancement,
> > >>>> etc)?
> > >>>> But, the OpenJPA performance is still 3 times slower than the
> > >>>> competitors?
> > >>>> In all of the scenarios? Or, just with this ManyToMany scenario? I
> > >>>> would
> > >>>> expect some overhead as compared to iBatis and/or straight JDBC, but
> > >>>> OpenJPA
> > >>>> should be competitive (and beat) the Hibernates and EclipseLinks...
> > >>>> Very
> > >>>> frustrating. When we do our comparisons with the industry
> benchmarks
> > >>>> (Trade
> > >>>> and SpecJApp), OpenJPA is extremely competitive.
> > >>>>
> > >>>> I have not closely examined your benchmark project, so I don't know
> > how
> > >>>> it
> > >>>> compares to Trade and/or SpecJApp work loads. Any thoughts on this
> > >>>> topic?
> > >>>>
> > >>>> One other thought... Just to prove that the enhancement processing
> is
> > >>>> being
> > >>>> done and you're not falling into the sub-classing support, could you
> > >>>> run
> > >>>> with the following property? This will cause your application to
> > >>>> error-off
> > >>>> if your Entities are not byte-code enhanced. We will not fall into
> > the
> > >>>> sub-classing support which greatly affects the performance.
> > >>>>
> > >>>> <property name="openjpa.RuntimeUnenhancedClasses"
> > >>>> value="warn"/>
> > >>>>
> > >>>> It really seems that you are trying to do a fair comparison, and I
> > >>>> greatly
> > >>>> appreciate your efforts. The last time one of these comparisons was
> > >>>> posted,
> > >>>> the benchmark code and process was flawed. So, I am pleased to see
> > the
> > >>>> efforts associated with this exercise.
> > >>>>
> > >>>> Our performance lead is out having a baby, so we haven't been able
> to
> > >>>> dig
> > >>>> into your benchmark to the extent that we would like. If we can
> > verify
> > >>>> that
> > >>>> the enhancement processing is happening, that would be good input.
> > >>>> Thanks
> > >>>> for your patience. What kind of timeframe are you under for posting
> > >>>> this
> > >>>> benchmark?
> > >>>>
> > >>>> Thanks,
> > >>>> Kevin
> > >>>>
> > >>>> On Thu, Mar 19, 2009 at 9:05 AM, Shubbis <marius.jo...@broadpark.no
> >
> > >>>> wrote:
> > >>>>
> > >>>>
> > >>>>
> > >>>>> Since we decided to go with vanilla installations of alle the
> > >>>>> frameworks
> > >>>>> we
> > >>>>> have not added the connection pool feature to OpenJPA, until now.
> > >>>>>
> > >>>>> The results are sadly not that great. Yes, it's faster and it
> doesn't
> > >>>>> run
> > >>>>> out of connections like before, BUT it's still 3, yes, -three-
> times
> > >>>>> slower
> > >>>>> than Hibernate, EclipseLink, iBatis and regular JDBC when
> persisting
> > >>>>> entities with many relations.
> > >>>>>
> > >>>>> Clearly this is not the kind of results I was hoping for and I'm
> > quite
> > >>>>> perplexed as to what to do.
> > >>>>>
> > >>>>> Shubbis
> > >>>>>
> > >>>>>
> > >>>>> Nitish Kumar wrote:
> > >>>>>
> > >>>>>
> > >>>>>> Hi subbis,
> > >>>>>> If I let the iteration loop over 5000, I get that exception,
> It
> > >>>>>> seems (I am not sure) openjpa is creating a new connection and
> after
> > >>>>>> a
> > >>>>>> while mysql runs out of connection. I tried the same code and
> > >>>>>> iteration
> > >>>>>> loop with a connection pool and it works fine. That should get you
> > >>>>>> moving as of now, till someone from Open JPA team looks into the
> > >>>>>> issue.
> > >>>>>>
> > >>>>>> Thanks and Regards,
> > >>>>>> Nitish Kumar
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>> --
> > >>>>> View this message in context:
> > >>>>>
> > >>>>>
> >
> http://n2.nabble.com/Slow-performance-with-OpenJPA-when-selecting-from-a-ManyToMany-relation.-tp2466994p2503084.html
> > >>>>> Sent from the OpenJPA Users mailing list archive at Nabble.com.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> > >
> >
> > --
> > View this message in context:
> >
> http://n2.nabble.com/Slow-performance-with-OpenJPA-when-selecting-from-a-ManyToMany-relation.-tp2466994p2516988.html
> > Sent from the OpenJPA Users mailing list archive at Nabble.com.
> >
> >
>
>
>
> --
> View this message in context:
> http://n2.nabble.com/Slow-performance-with-OpenJPA-when-selecting-from-a-ManyToMany-relation.-tp2466994p2532837.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
>
