On 4/9/14, 11:48 AM, Gregory Szorc wrote:
I feel a lot of people just shrug shoulders and allow the test to be disabled (I'm guilty of it as much as anyone). From my perspective, it's difficult to convince the powers at be that fixing intermittent failures (that have been successfully swept under a rug and are out of sight and out of mind) is more important than implementing some shiny new feature (that shows up on an official goals list). I feel we all need to treat failing tests with more urgency. The engineering culture is not currently favoring that.
How can we realign our testing priorities, short of some management decree? Sheriffs have had to disable entire test suites, but even that has not motivated test owners to fix their intermittent tests. Part of the problem, I understand, is that many tests have no owner, so the responsibility for even debugging test failures is not clear.
How much effort do we want to spend creating tools to manage and rerun known intermittent tests compared to diagnosing the root problems? Earlier in this thread, Ehsan said that, once upon a time, we had no intermittent tests. Someone else suggested (half-jokingly, I assume :) that requiring a 100% green TBPL before landing on inbound would be an effective motivator to fix intermittent tests.
chris _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
