On Thu, Aug 1, 2013 at 1:06 PM, Clint Talbert <ctalb...@mozilla.com> wrote: > > All that said, this thread is much bigger than one automation bug. > There are several core issues here: > * Continual investigation into intermittent failures as they come up. If we > don't continue prioritizing these fixes, things escalate into disasters very > quickly. The most important one right now is the one preventing the Gaia > Unit tests from being un-hidden across all trees: bug 898512
This comes up fairly often, and I'm sure rightfully so. Every time I tell people to let me know. So far I've only been pinged once and I was able to find resources to attack the problems. I haven't heard anything since so I had assumed that we were in a good state. So again, please do let me know when this happens. I'm happy to create a hot-line escalation path as to ensure that any intermittent bug gets attention within 24 hours. Testing is one of *the* top priorities for v1.2, and so we should be able to get attention on intermittent tests. > * Turning on more testing using existing frameworks - the Gaia Unit tests > are nearly ready to go, the Gaia UI tests are next in line (bug 897569), and > the Gaia "integration" (Lal's JS framework) tests are after that. This will > help ensure we capture errors as they surface more quickly, but catching > errors doesn't do us any good if we don't prioritize fixing them. And most importantly, getting these landed means that we can institute a policy of that all new patches should land with tests. And that we can start writing tests for the backlog of things that has already landed. > * Identifying where we are lacking test coverage and testability and adding > tests and testability into the product in those areas. Agreed. All teams should identify areas that are lacking tests and add tasks to the backlog to add tests to such areas. > * Having QA step in and do manual and automated tests of master on devices > will continue to help (and it already is). QA is adding bugs they find to > the B2G-on-central tracking bug: > https://bugzilla.mozilla.org/showdependencytree.cgi?id=884399&hide_resolved=1 Agreed! / Jonas _______________________________________________ dev-b2g mailing list dev-b2g@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-b2g
