Awesome! I think this would be great to have for the integration tests in B2G as well.
/ Jonas On Fri, Dec 12, 2014 at 10:34 AM, Ehsan Akhgari <ehsan.akhg...@gmail.com> wrote: > We had a session on intermittent test failures in Portland < > https://etherpad.mozilla.org/ateam-pdx-intermittent-oranges>, and one of > the things that we discussed was adding analyses to our test suites that > detect known bad test writing practices < > https://developer.mozilla.org/en-US/docs/Mozilla/QA/Avoiding_intermittent_oranges> > and fails the offending tests. As part of that effort, I just landed the > patch to bug 649012 which enables this analysis on mochitest-plain. > > What this means for you as a test author: > * Using setTimeout(foo, N); where N is not 0 will cause your tests to > fail. You should double check why you're using such timeouts. They are > almost never necessary. > * If you have a legitimate reason for using timeouts like this (such as if > your test needs to wait a bit to make sure an event doesn't happen in the > future, and there is no other way of checking that), you should call > SimpleTest.requestFlakyTimeout("reason");. The argument to the function is > a string which is meant to document why you need to use these kinds of > timeouts, and why that doesn't lead into intermittent test failures. > > What this means for you as a code reviewer: > * If you see a patch which uses SimpleTest.requestFlakyTimeout, please > review the usage of timeouts carefully and make sure they are safe in the > test, and the reason argument passed to that function correctly documents > it. > > What this means for you as a module owner/peer: > * I didn't have time to triage all of the existing tests that this analysis > caught, so I have annotated them all with > |SimpleTest.requestFlakyTimeout("untriaged");|. See < > https://hg.mozilla.org/integration/mozilla-inbound/rev/fa8bd3194aca> for > examples. Please go through the code in your module and try to triage > these tests, and either fix the timeout usages, or if they are indeed OK > document the reason in requestFlakyTimeout's argument. I had to do this in > order to prevent new bad tests from going in as soon as possible. > > I have turned this on only for mochitest-plain for now, and am planning to > do the same for mochitest-chrome/a11y/browser-chrome very soon as well. > > Cheers, > -- > Ehsan > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
