<quote name="Nikolas Everett" date="2013-09-19" time="16:57:35 -0400"> > On Thu, Sep 19, 2013 at 4:55 PM, Greg Grossmeier <[email protected]> wrote: > > <quote name="Rachel Thomas" date="2013-09-19" time="16:26:39 -0400"> > >> In that case, I can see why Zeljko removed them. I guess the next step > >> would be to think of what deliberate tests we should run against > >> production. > > > > +1 > > > > What are good canaries for larger issues on production? > > Another way to think about this is what are things that we really > really really want to notice fast if they break. For instance, the > main page of any wiki stops working, we should know.
My understanding/assumption is that these aren't run very frequently; something like 3 times/day. What's the phrase? Any sufficiently advanced testing is indistinguishable from monitoring? The other way around? whatever :) I think I'd focus on: What are the features that are important but aren't easily monitored through ganglia/whatever? What set of tests can we use to ensure there aren't oddities on real production sites that aren't there on the fake production sites like test2? Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D | _______________________________________________ QA mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/qa
