Hi,
Here's a few more simple benchmark results to show where we are:
Apache Jackrabbit Oak 0.7-SNAPSHOT
# ReadPropertyTest min 10% 50% 90% max N
Jackrabbit31 31 33 92 1211470
Oak-Default
Hi,
Thanks! The SmallFileWriteTest is quite slow for Oak-Mongo: 6 times slower
than Jackrabbit on average, as far as I see. It seems to be, at least
partially, a problem of the AbstractBlobStore. I will have a look.
Regards,
Thomas
On 3/27/13 10:41 AM, "Jukka Zitting" wrote:
>Hi,
>
>Here's
Hi,
On Wed, Mar 27, 2013 at 11:41 AM, Jukka Zitting wrote:
> # ReadPropertyTest min 10% 50% 90% max
> N
> Jackrabbit31 31 33 92 121
> 1470
> Oak-Default 56 58 61 120
On 27.3.13 11:54, Jukka Zitting wrote:
Hi,
Do we need to explicitly validate all paths that get passed to us?
Especially in cases like getProperty(), where in the vast majority of
the cases the given path matches an existing property (whose path by
definition is valid), it would make more s
Hi,
On Wed, Mar 27, 2013 at 2:12 PM, Michael Dürig wrote:
> IIUC you propose to not validate paths in the read case but rely on the
> downstream code to fail. Might be worth a try. However we'd need different
> path parsing then for the read an the write case since circumventing path
> validation
On 27.3.13 12:21, Jukka Zitting wrote:
Hi,
On Wed, Mar 27, 2013 at 2:12 PM, Michael Dürig wrote:
IIUC you propose to not validate paths in the read case but rely on the
downstream code to fail. Might be worth a try. However we'd need different
path parsing then for the read an the write case
Hi,
On Wed, Mar 27, 2013 at 2:32 PM, Michael Dürig wrote:
> That's right. The easiest thing is to try it out, remove pre-emptive path
> validation and see what breaks. I have the vague memory that there were some
> overly picky TCK tests which required us to put this upfront validation in.
> Howe
Hi,
On Wed, Mar 27, 2013 at 1:54 PM, Jukka Zitting wrote:
> Quick benchmarking of the Oak-Default run shows
> NamePathMapperImpl.getOakPath() calling JcrPathParser.validate()
> taking about 20% of the time in this test.
Updated numbers after the latest OAK-108 change:
# ReadPropertyTest
Hi,
On Wed, Mar 27, 2013 at 11:41 AM, Jukka Zitting wrote:
> Here's a few more simple benchmark results to show where we are:
Some notes to help read and produce benchmark results like these.
The oak-run jar that you can find under oak-run/target has a
"benchmark" mode that produces these resul
On 27.3.13 14:41, Jukka Zitting wrote:
Profiling the getProperty calls shows the following distribution of time spent:
NodeImpl.getProperty()
61% NodeDelegate.getProperty() via perform()
31% ItemImpl.isStale() via checkStatus()
8% other stuff
The status check would
On 27.3.13 14:41, Jukka Zitting wrote:
Drilling down to the NodeDelegate.getProperty() method, we have the
following distribution of time:
NodeDelegate.getProperty()
95% NodeDelegate.getChildLocation()
5% TreeImpl.internalGetProperty() via NodeLocation.getProperty()
See wh
On 28.3.13 15:19, Angela Schreiber wrote:
hi michael
With the resolution of OAK-690, I made tree instances stable across save
and refresh operations.
does that mean that the AuthorizableImpl could hold a Tree instance
instead of re-accessing it over and over again using a lookup by id?
if t
oh... already fixed it before lunch and committed modifications
before re-checking mails (using OAK-690 and an appropriate comment).
thanks anyway... test passed and i assume it's fine. the internal method
AuthorizableImpl#getTree will now return the Tree associated
with the authorizable as lon
Hi,
On Wed, Mar 27, 2013 at 11:41 AM, Jukka Zitting wrote:
> Here's a few more simple benchmark results to show where we are:
Updated numbers with latest Oak:
# ReadPropertyTest min 10% 50% 90% max N
Jackrabbit34 35 3
Hello,
I'm interested in estimating performance (and load) impacts of ACL
checking on read access. I'm specifically interested in a comparison where
paths like /a, /a/b, /a/b/c, /a/.../y/z are accessed, and ACL has to be
evaluated "upwards" on the path. Since such a test is more high-level and
may
Hi,
On Mon, Apr 29, 2013 at 10:17 AM, Lukas Eder wrote:
> Are there any test results available with respect to ACL, comparing
> Jackrabbit with Oak?
Not yet. See the o.a.j.oak.benchmark package in oak-run for some of
the existing benchmarks I've been using so far. It should be fairly
straightfor
Hi,
On Fri, Apr 26, 2013 at 2:12 PM, Jukka Zitting wrote:
> On Wed, Mar 27, 2013 at 11:41 AM, Jukka Zitting
> wrote:
>> Here's a few more simple benchmark results to show where we are:
>
> Updated numbers with latest Oak:
And another one:
Apache Jackrabbit Oak 0.9-SNAPSHOT
# ReadPrope
ginal Message-
From: Jukka Zitting [mailto:jukka.zitt...@gmail.com]
Sent: Friday, May 31, 2013 8:14 AM
To: Oak devs
Subject: Re: Some more benchmarks
Hi,
On Fri, Apr 26, 2013 at 2:12 PM, Jukka Zitting wrote:
> On Wed, Mar 27, 2013 at 11:41 AM, Jukka Zitting
> wrote:
>> Here
Hi,
On Fri, May 31, 2013 at 3:52 PM, Michael C. Moore wrote:
> Can you briefly explain the test results or point me to a wiki or link that
> has the explanation?
I just committed the description to a README file, see
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-run/README.md
BR,
Ju
Great, thanks!
-Original Message-
From: Jukka Zitting [mailto:jukka.zitt...@gmail.com]
Sent: Friday, May 31, 2013 9:28 AM
To: Oak devs
Subject: Re: Some more benchmarks
Hi,
On Fri, May 31, 2013 at 3:52 PM, Michael C. Moore wrote:
> Can you briefly explain the test results or point
Hi,
A bit weird is, when I run the tests separately I get different numbers:
java -mx1g -jar target/oak-run-0.9-SNAPSHOT.jar benchmark
SmallFileReadTest Oak-Tar
# SmallFileReadTest min 10% 50% 90% max
N
Oak-Tar 53 54 55 57
Hi,
On Mon, Jun 3, 2013 at 11:09 AM, Thomas Mueller wrote:
> A bit weird is, when I run the tests separately I get different numbers:
The results depend on the hardware you're using, so in general numbers
from two different environments are not directly comparable.
> In your case, the N was 304
Hi,
I was not talking about differences in hardware. I know using different
hardware will result in different numbers.
I was worried about results would be different if you run one test alone
versus if you run all tests. That would indicate a problem in the
benchmark (framework) itself.
But luck
Hi,
On Mon, Jun 3, 2013 at 12:51 PM, Thomas Mueller wrote:
> I was not talking about differences in hardware. I know using different
> hardware will result in different numbers.
>
> I was worried about results would be different if you run one test alone
> versus if you run all tests. That would
Hi,
>At least the TarMK should have no problems with the 100 child nodes
>(see the Wikipedia import test results :-).
Yes, I also thought 100 child nodes shouldn't be a problem. The profiling
data I have so far doesn't show a clear bottleneck. I really wonder what
the problem is in this case.
Re
Hi,
On Fri, May 31, 2013 at 3:14 PM, Jukka Zitting wrote:
> It looks like we have a performance regression in ReadPropertyTest.
> Quick profiling shows a lot of the time seems to be going to
> MemoryNodeBuilder$ConnectedHead.update(), which is weird since we're
> only reading and thus the related
Hi,
On Fri, May 31, 2013 at 3:14 PM, Jukka Zitting wrote:
> On Fri, Apr 26, 2013 at 2:12 PM, Jukka Zitting
> wrote:
>> On Wed, Mar 27, 2013 at 11:41 AM, Jukka Zitting
>> wrote:
>>> Here's a few more simple benchmark results to show where we are:
>>
>> Updated numbers with latest Oak:
>
> And
Hi,
Thanks a lot!
> I've only included the 90th percentile
I usually look at "N" first :-)
There is one strange result: SmallFileWriteTest; Oak-Segment: 90%=257,
max=14763 - Maybe the warmup phase is too short, or the test isn't that
great?
As for SmallFileReadTest and SmallFileWriteTest with
Hi,
On Wed, Jul 3, 2013 at 11:22 AM, Thomas Mueller wrote:
>> I've only included the 90th percentile
>
> I usually look at "N" first :-)
It's also a good measure. I like the 90th percentile better as it
filters out outliers that may otherwise weight pretty heavily on the
total or average executi
Hi,
On Wed, Jul 3, 2013 at 11:54 AM, Jukka Zitting wrote:
> On Wed, Jul 3, 2013 at 11:22 AM, Thomas Mueller wrote:
>> I usually look at "N" first :-)
>
> It's also a good measure.
Actually not that good, as only the lower limit on the amount of time
over which those N iterations happen is defin
Hi,
>I don't know what's dragging the performance in the SmallFileReadTest,
I think it's because all binaries are loaded from the backend (no
caching). I bumped the blob cache size from 8 MB to 16 MB, let's see if
this helps.
Regards,
Thomas
Hi,
On Wed, Jul 3, 2013 at 5:34 PM, Thomas Mueller wrote:
> I think it's because all binaries are loaded from the backend (no
> caching). I bumped the blob cache size from 8 MB to 16 MB, let's see if
> this helps.
Yes, that could be it (I'll run the tests again soon to confirm). The
working set
Hi,
On Thu, Jul 4, 2013 at 1:12 PM, Jukka Zitting wrote:
> On Wed, Jul 3, 2013 at 5:34 PM, Thomas Mueller wrote:
>> I think it's because all binaries are loaded from the backend (no
>> caching). I bumped the blob cache size from 8 MB to 16 MB, let's see if
>> this helps.
>
> Yes, that could be i
Hi,
Updating this thread with the latest numbers. No major changes on
these benchmarks:
# ReadPropertyTest 90%
Jackrabbit48
Oak-Default 39
Oak-Mongo 41
Oak-Segment 41
Hi,
On Tue, Sep 24, 2013 at 8:08 PM, Jukka Zitting wrote:
> I'll add a few more benchmarks to my test script.
Here are results for three more benchmarks:
# SetPropertyTest 90%
Jackrabbit 740
Oak-Default 916
Oak-Mongo
Hi,
On Tue, Sep 24, 2013 at 10:47 PM, Jukka Zitting wrote:
> The concurrent read and read/write test cases look like more attention
> is needed on the test code, as it's currently hard to interpret the
> results. I'll see what I can do there.
It turns out most of the reported time was going to l
Hi,
On Tue, Sep 24, 2013 at 11:19 PM, Jukka Zitting wrote:
> On Tue, Sep 24, 2013 at 10:47 PM, Jukka Zitting
> wrote:
>> The concurrent read and read/write test cases look like more attention
>> is needed on the test code, as it's currently hard to interpret the
>> results. I'll see what I can
Hi,
I'm resurrecting this thread with some new findings. I re-ran many of
the benchmarks we've been following, pitting Jackrabbit 2.8.0 against
Oak 1.0.1 with TarMK. The results look pretty nice:
Summary (90%, lower is better)
Benchmark Jackrabbit Oak-Tar
--
Hi,
On Tue, Jul 1, 2014 at 9:38 AM, Jukka Zitting wrote:
> I also tried including MongoMK results, but the benchmark got stuck in
> ConcurrentReadTest. I'll re-try today and will file a bug if I can
> reproduce the problem.
I guess it was a transient problem. Here are the results with
Oak-Mongo
On Tue, Jul 1, 2014 at 8:37 PM, Jukka Zitting wrote:
> Hi,
>
> On Tue, Jul 1, 2014 at 9:38 AM, Jukka Zitting wrote:
>> I also tried including MongoMK results, but the benchmark got stuck in
>> ConcurrentReadTest. I'll re-try today and will file a bug if I can
>> reproduce the problem.
>
> I guess
40 matches
Mail list logo