In general I agree but as I have reported, it looks like in this case, even single-threaded write operations are taking a considerable longer time. The load is small and although my own junit uses store releasing, our client, in their performance runs, don't ever use store release. I cannot prove this just yet, but it smells like simple (single-threaded) write operations are slower.
However, in order to test the regression, I'd need a 2.7.1 snapshot from May 15. I checked the snapshots pages and they are gone. How do you suggest to get the older build back? (I don't want to use 0.9.0 because there are too many differences -> based on our experience, the regression is very recent) Simon From: Andy Seaborne <[email protected]> To: [email protected] Date: 06/14/2012 07:57 AM Subject: Re: concurrency and slower results On 14/06/12 11:52, Laurent Pellegrino wrote: > It is strongly linked to our application. I think it would be simpler > to write new test cases to include them with others. Ideally, the > results (e.g. the throughput) could be plotted automatically by > Jenkins with the Plot plugin > (https://wiki.jenkins-ci.org/display/JENKINS/Plot+Plugin) to easily > check differences between builds. > > Laurent What would be helpful is to know the queries, the characteristics of the data and the use model (e.g. update / query mix, total load, releasing stores etc). Otherwise, it's the proverbial hunting for a needle in a haystack. Were other benchmarks affected? If not, what's the key difference? Andy > > On Wed, Jun 13, 2012 at 10:57 PM, Andy Seaborne<[email protected]> wrote: >> On 13/06/12 16:06, Laurent Pellegrino wrote: >>> >>> Hello, >>> >>> I also noticed the same thing by updating jena-tdb only. I switched >>> from version 0.9.0-incubating to snapshots on 6 June and I noticed >>> around ~50% decrease on one of our benchmark which is read/write >>> intensive. >>> >>> Kind Regards, >>> >>> Laurent >> >> >> Can you share the benchmark system? >> >> Andy >> >> >>> >>> On Wed, Jun 13, 2012 at 4:33 PM, Simon Helsen<[email protected]> wrote: >>>> >>>> before you ask, of course, we are looking into whether this is something >>>> on our end. We only made very few changes, but we should be able to >>>> isolate this. I'll report on that once I know for sure. My question was >>>> more whether anyone knew of any changes in tx that could affect >>>> concurrent >>>> performance negatively since May 15 >>>> >>>> thanks >>>> >>>> Simon >>>> >>>> >>>> >>>> >>>> >>>> From: >>>> Simon Helsen/Toronto/IBM@IBMCA >>>> To: >>>> [email protected] >>>> Date: >>>> 06/13/2012 10:09 AM >>>> Subject: >>>> concurrency and slower results >>>> >>>> >>>> >>>> hi everyone, Andy especially, >>>> >>>> it looks like some of the changes to TDB since May 15 have negatively >>>> affected the concurrent behavior of Tx. To clarify, we are doing some >>>> serious internal performance testing on a product which is based on >>>> Jena/TDB. We initially did these runs to compare Tx (2.7.1 snapshot of >>>> May >>>> >>>> 15) with our old TDB implementation (which used a conservative locking >>>> model). The results were good, especially with long-running/expensive >>>> index update operations. >>>> >>>> Now, I had asked the performance team to rerun their tests with a new >>>> build of the product based on the release candidate (of last weekend) and >>>> they are reporting a concerning degradation. Compared to the May 15 TDB, >>>> they are reporting 12-15% decrease in query performance when doing a >>>> "light update load" and about 40% query and 40% update performance >>>> degradation when doing a "heave update load". >>>> >>>> That is quite a serious regression. The question is what changed since >>>> May >>>> >>>> 15 that could have such a bad effect on performance. The tests included >>>> only 25 concurrent users on about 15 million triples. Yet, I wonder if >>>> some of the recent lock changes (see JENA-252) are responsible >>>> >>>> thanks >>>> >>>> Simon >>>> >>>> >>>> >>
