This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 11ed566c7689f9363cb803a61eb9765723b634d2
Author: Benoit Tellier <btell...@linagora.com>
AuthorDate: Wed Dec 4 18:23:37 2019 +0700
JAMES-2919 Validate JMAP partial read ADR after performance tests
---
src/adr/0012-jmap-partial-reads.md | 23 ++++++++++++++++-------
src/adr/0013-precompute-jmap-preview.md | 19 ++++++++++++++-----
2 files changed, 30 insertions(+), 12 deletions(-)
diff --git a/src/adr/0012-jmap-partial-reads.md
b/src/adr/0012-jmap-partial-reads.md
index 310cd2e..ef945b6 100644
--- a/src/adr/0012-jmap-partial-reads.md
+++ b/src/adr/0012-jmap-partial-reads.md
@@ -4,9 +4,7 @@ Date: 2019-10-09
## Status
-Proposed
-
-Adoption needs to be backed by some performance tests.
+Accepted
## Context
@@ -29,10 +27,21 @@ Some performance tests will be run in order to evaluate the
improvements.
## Consequences
-GetMessages with a limited set of requested properties will no longer result
necessarily in full database message read. We
-thus expect a significant improvement, for instance when only metadata are
requested.
-
-In case of a less than 5% improvement, the code will not be added to the
codebase and the proposal will get the status 'rejected'.
+GetMessages with a limited set of requested properties no longer result
necessarily in full database message read. We
+thus have a significant improvement, for instance when only metadata are
requested.
+
+Given the following scenario played by 5000 users per hour (constant rate)
+ - Authenticate
+ - List mailboxes
+ - List messages in one of their mailboxes
+ - Get 10 times the mailboxIds and keywords of the given messages
+
+We went from:
+ - A 20% failure and timeout rate before this change to no failure
+ - Mean time for GetMessages went from 27 159 ms to 27 ms (1000 time
improvment), for all operation from
+ 27 591 ms to 60 ms (460 time improvment)
+ - P99 is a metric that did not make sense because the initial simulation
exceeded Gatling (the performance measuring tool
+ we use) timeout (60s) at the p50 percentile. After this proposal p99 for the
entire scenario is of 1 383 ms
## References
diff --git a/src/adr/0013-precompute-jmap-preview.md
b/src/adr/0013-precompute-jmap-preview.md
index e683ab2..8a1092f 100644
--- a/src/adr/0013-precompute-jmap-preview.md
+++ b/src/adr/0013-precompute-jmap-preview.md
@@ -4,9 +4,7 @@ Date: 2019-10-09
## Status
-Proposed
-
-Adoption needs to be backed by some performance tests.
+Accepted
## Context
@@ -35,9 +33,20 @@ Some performance tests will be run in order to evaluate the
improvements.
## Consequences
-We expect a huge performance enhancement for JMAP clients relying on preview
for listing mails.
+Given the following scenario played by 2500 users per hour (constant rate)
+ - Authenticate
+ - List mailboxes
+ - List messages in one of their mailboxes
+ - Get 8 times the properties expected to be fast to fetch with JMAP
+
+We went from:
+ - A 7% failure and timeout rate before this change to almost no failure
+ - Mean time for GetMessages went from 9 710 ms to 434 ms (22 time
improvment), for all operation from
+ 12 802 ms to 407 ms (31 time improvment)
+ - P99 is a metric that did not make sense because the initial simulation
exceeded Gatling (the performance measuring tool
+ we use) timeout (60s) at the p95 percentile. After this proposal p99 for the
entire scenario is of 1 747 ms
-In case of a less than 5% improvement, the code will not be added to the
codebase and the proposal will get the status 'rejected'.
+As such, this changeset significantly increases the JMAP performance.
## References
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
