Hi,
On Wed, Jul 3, 2013 at 11:22 AM, Thomas Mueller <muel...@adobe.com> wrote:
>> I've only included the 90th percentile
>
> I usually look at "N" first :-)
It's also a good measure. I like the 90th percentile better as it
filters out outliers that may otherwise weight pretty heavily on the
total or average execution time. Of course, as you note below, it's
good to pay attention also to such cases.
> There is one strange result: SmallFileWriteTest; Oak-Segment: 90%=257,
> max=14763 - Maybe the warmup phase is too short, or the test isn't that
> great?
Good catch. I ran the benchmark a few more times, and the max time is
always pretty high. It shouldn't be about the warmup phase, as the
time in this test should be governed by the blob I/O. I'll try to
figure out what's causing such worst case behavior.
> As for SmallFileReadTest and SmallFileWriteTest with Oak-Mongo: I think I
> know what the problem is; it doesn't seem to be related to BLOB handling
> at all (actually performance is the same without the BLOB), but partially
> related to the "split documents" that should be added in the near future.
> Also, it seems to be partially related to what the test does (repeatedly
> adding and removing the same nodes).
Right. The semantics of the SmallFileWriteTest should be the same if
the test root name was different for each test iteration, which should
avoid the slit document edge case. I adjusted the test (see patch
below), and the numbers do look a bit better but not radically so:
# SmallFileWriteTest min 10% 50% 90% max N
Oak-Mongo 577 591 708 1198 1585 33
I don't know what's dragging the performance in the SmallFileReadTest,
as there the nodes are created just once at the beginning of the
benchmark.
BR,
Jukka Zitting
----
diff --git
a/oak-run/src/main/java/org/apache/jackrabbit/oak/benchmark/SmallFileWriteTest.java
b/oak-run/src/main/java/org/apache/jackrabbit/oak/benchmark/SmallFileWriteTest.java
index 7d15b00..c5f2ec8 100644
---
a/oak-run/src/main/java/org/apache/jackrabbit/oak/benchmark/SmallFileWriteTest.java
+++
b/oak-run/src/main/java/org/apache/jackrabbit/oak/benchmark/SmallFileWriteTest.java
@@ -32,6 +32,8 @@ public class SmallFileWriteTest extends AbstractTest {
private Node root;
+ private long count = 0;
+
@Override
public void beforeSuite() throws RepositoryException {
session = loginWriter();
@@ -39,7 +41,7 @@ public class SmallFileWriteTest extends AbstractTest {
@Override
public void beforeTest() throws RepositoryException {
- root = session.getRootNode().addNode("SmallFileWriteTest",
"nt:folder");
+ root = session.getRootNode().addNode("SmallFileWriteTest" +
count++, "nt:folder");
session.save();
}
