> mode against production servers causes perceptible degradation on FxA
performance
To be clear, there is no degradation in performance. But it really skews
some metrics because of however this test executes, and I do not need
this noise.
Karl, can we chat about how this is run?
John
On 08/08/16 14:52, Karl Thiessen wrote:
To introduce this conversation to a wider audience:
TPS (Testing and Performance for Sync) is a comprehensive automated
test suite developed in a collaboration between the A-Team (now
renamed the Engineering Productivity Team) and the Sync team to test
Sync in ways that are not allowed in Tier 1 testing, namely network
requests to a live server.
Since Sync and FxA are tightly coupled, running TPS in a "continuous
testing" mode against production servers causes perceptible
degradation on FxA performance. So let's not do that.
Also, TPS was originally conceived as a method of testing the Sync
client code that lives in Firefox against "known-good" Sync server,
i.e. the production Sync cluster. But I know of no reason why it
could not be used the other way around, to test new changes to the
Sync server with "known-good" (i.e. released) Firefox clients,
potentially both desktop and mobile.
To do this, we need two parts:
1. A way to point the TPS suite at a non-production Sync server (this
is https://bugzilla.mozilla.org/show_bug.cgi?id=1293426)
and
2. An automated/CI way to run intensive TPS tests against Stage.
(this is being discussed in
https://bugzilla.mozilla.org/show_bug.cgi?id=1219456, but will
probably be broken out into a new bug soonish.)
If anyone has opinions, thoughts, caveats, vigourous objections, or
other sorts of comments on this idea, please feel free to pipe up in
one of the referenced bugs, or continue the discussion here.
Thanks very much,
--KT.
_______________________________________________
Sync-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/sync-dev
d
_______________________________________________
Sync-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/sync-dev
