Running TPS against stage instead of production is very appealing to me, but it does beg the question: what kind of SLA will be required for the staging Sync cluster that it points at? If it's not beyond best-effort we'll be fine.
--Bob On Mon, Aug 8, 2016 at 2:52 PM, Karl Thiessen <[email protected]> wrote: > To introduce this conversation to a wider audience: > > TPS (Testing and Performance for Sync) is a comprehensive automated test > suite developed in a collaboration between the A-Team (now renamed the > Engineering Productivity Team) and the Sync team to test Sync in ways that > are not allowed in Tier 1 testing, namely network requests to a live server. > > Since Sync and FxA are tightly coupled, running TPS in a "continuous > testing" mode against production servers causes perceptible degradation on > FxA performance. So let's not do that. > > Also, TPS was originally conceived as a method of testing the Sync client > code that lives in Firefox against "known-good" Sync server, i.e. the > production Sync cluster. But I know of no reason why it could not be > used the other way around, to test new changes to the Sync server with > "known-good" (i.e. released) Firefox clients, potentially both desktop and > mobile. > > To do this, we need two parts: > > 1. A way to point the TPS suite at a non-production Sync server (this is > https://bugzilla.mozilla.org/show_bug.cgi?id=1293426) > > and > > 2. An automated/CI way to run intensive TPS tests against Stage. (this is > being discussed in https://bugzilla.mozilla.org/show_bug.cgi?id=1219456, > but will probably be broken out into a new bug soonish.) > > If anyone has opinions, thoughts, caveats, vigourous objections, or other > sorts of comments on this idea, please feel free to pipe up in one of the > referenced bugs, or continue the discussion here. > > Thanks very much, > --KT. > > _______________________________________________ > Sync-dev mailing list > [email protected] > https://mail.mozilla.org/listinfo/sync-dev > >
_______________________________________________ Sync-dev mailing list [email protected] https://mail.mozilla.org/listinfo/sync-dev
