Re: Reduction of intermittent failures for Marionette tests

2016-11-09 Thread Lawrence Mandel
Thank you for your focus on intermittent Marionette tests Henrik. Great
results!

Lawrence

On Wed, Nov 9, 2016 at 4:16 AM, Henrik Skupin  wrote:

> Hi all,
>
> Over the past months I tried to always have a look on newly filed
> intermittent failures for Marionette tests on Bugzilla. I tried to
> triage them early and to bring those up to interested parties. This was
> working great for a couple of scenarios like the massive amount of hangs
> and crashes for e10s tests. But I also did a lot of work myself to
> stabilize the tests by making sure we correctly handle:
>
> * The external process controlled by Marionette (tracking bug 1283906)
> * Chrome windows and tabs (bug 1259055)
>
> With the latest landing of the patch on bug 1299216 to improve the crash
> handling of Marionette, which also adds support for content crashes, I
> will mark this work as done.
>
> The results of all this work you can already see on integration
> branches. Marionette tests have a very low intermittent failure rate and
> fail only a handful of times over a day.
>
> I hope that we can keep that rate for a while now to make the sheriffs
> happy.
>
> --
> Henrik Skupin
> Senior Software Engineer
> Mozilla Corporation
> ___
> tools mailing list
> to...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/tools
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


Reduction of intermittent failures for Marionette tests

2016-11-09 Thread Henrik Skupin
Hi all,

Over the past months I tried to always have a look on newly filed
intermittent failures for Marionette tests on Bugzilla. I tried to
triage them early and to bring those up to interested parties. This was
working great for a couple of scenarios like the massive amount of hangs
and crashes for e10s tests. But I also did a lot of work myself to
stabilize the tests by making sure we correctly handle:

* The external process controlled by Marionette (tracking bug 1283906)
* Chrome windows and tabs (bug 1259055)

With the latest landing of the patch on bug 1299216 to improve the crash
handling of Marionette, which also adds support for content crashes, I
will mark this work as done.

The results of all this work you can already see on integration
branches. Marionette tests have a very low intermittent failure rate and
fail only a handful of times over a day.

I hope that we can keep that rate for a while now to make the sheriffs
happy.

-- 
Henrik Skupin
Senior Software Engineer
Mozilla Corporation
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform