On 14/06/12 11:52, Laurent Pellegrino wrote:
It is strongly linked to our application. I think it would be simpler
to write new test cases to include them with others. Ideally, the
results (e.g. the throughput) could be plotted automatically by
Jenkins with the Plot plugin
(https://wiki.jenkins-ci.org/display/JENKINS/Plot+Plugin) to easily
check differences between builds.
Laurent
What would be helpful is to know the queries, the characteristics of the
data and the use model (e.g. update / query mix, total load,
releasing stores etc). Otherwise, it's the proverbial hunting for a
needle in a haystack.
Were other benchmarks affected? If not, what's the key difference?
Andy
On Wed, Jun 13, 2012 at 10:57 PM, Andy Seaborne<[email protected]> wrote:
On 13/06/12 16:06, Laurent Pellegrino wrote:
Hello,
I also noticed the same thing by updating jena-tdb only. I switched
from version 0.9.0-incubating to snapshots on 6 June and I noticed
around ~50% decrease on one of our benchmark which is read/write
intensive.
Kind Regards,
Laurent
Can you share the benchmark system?
Andy
On Wed, Jun 13, 2012 at 4:33 PM, Simon Helsen<[email protected]> wrote:
before you ask, of course, we are looking into whether this is something
on our end. We only made very few changes, but we should be able to
isolate this. I'll report on that once I know for sure. My question was
more whether anyone knew of any changes in tx that could affect
concurrent
performance negatively since May 15
thanks
Simon
From:
Simon Helsen/Toronto/IBM@IBMCA
To:
[email protected]
Date:
06/13/2012 10:09 AM
Subject:
concurrency and slower results
hi everyone, Andy especially,
it looks like some of the changes to TDB since May 15 have negatively
affected the concurrent behavior of Tx. To clarify, we are doing some
serious internal performance testing on a product which is based on
Jena/TDB. We initially did these runs to compare Tx (2.7.1 snapshot of
May
15) with our old TDB implementation (which used a conservative locking
model). The results were good, especially with long-running/expensive
index update operations.
Now, I had asked the performance team to rerun their tests with a new
build of the product based on the release candidate (of last weekend) and
they are reporting a concerning degradation. Compared to the May 15 TDB,
they are reporting 12-15% decrease in query performance when doing a
"light update load" and about 40% query and 40% update performance
degradation when doing a "heave update load".
That is quite a serious regression. The question is what changed since
May
15 that could have such a bad effect on performance. The tests included
only 25 concurrent users on about 15 million triples. Yet, I wonder if
some of the recent lock changes (see JENA-252) are responsible
thanks
Simon
