Is there any effort under way to making the smoketests automated and run as part of our regular automation testing? The B2G QA team does a great job of identifying regressions and tracking down the regressing changeset, but AFAIK this is an entirely manual process that happens after the change has landed. Ideally we should catch this on try pushes or on landing though, and for that we need to automate the smoketests.
There have been a lot of complaints (and rightly so) about all sorts of B2G-breaking changesets landing. I myself have landed quite a few of them. I think it's unrealistic to expect every Gecko developer to run through all of the smoketests manually for every change they want to make (even just for the main devices/configurations we support). It's also unrealistic to expect them to reliably identify "high risk" changes for explicit pre-landing QA testing, because even small changes can break things badly on B2G given the variety of configurations we have there. I think the only reasonable long-term solution is to automate the smoketests, and I would like to know if there's any planned or in-progress effort to do that. Cheers, kats _______________________________________________ dev-b2g mailing list dev-b2g@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-b2g
