The mochitest, reftest, and xpcshell test harnesses now support a --verify option. For example:
mach mochitest docshell/test/test_anchor_scroll_after_document_open.html --verify In verify mode, the requested test is run multiple times, in various "modes", in hopes of quickly finding any intermittent failures. Once tests are complete, a summary of results is printed: ::: ::: Test verification summary for: ::: ::: docshell/test/test_anchor_scroll_after_document_open.html ::: ::: 1. Run each test 10 times in one browser. : Pass ::: 2. Run each test 5 times in a new browser each time. : Pass ::: 3. Run each test 10 times in one browser, in chaos mode. : Pass ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : Pass ::: ::: Test verification PASSED ::: There's no flexibility in the number of times the test is run in each mode and that's by design: I wanted a simple, standard way of checking "Is this test likely to produce a frequent intermittent failure"? Verify mode was developed for the test-verify task (announcement coming soon!) but it may also be a convenient addition to your local testing. More info at [1]. Bug and enhancement requests welcomed: please file bugs blocking bug 1357513. [1] https://developer.mozilla.org/en-US/docs/Test_Verification _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
