Re: [webkit-dev] Handling flaky layout-test failures in EWS

> 5 дек. 2019 г., в 7:58 AM, Aakash Jain написал(а): > > > What is the current behavior when a patch introduces substantial flakiness? > > For the patch which introduces substantial flakiness (and no consistent > failure in EWS), prior to r253049 (which I landed two days back), the patch > w

Re: [webkit-dev] Handling flaky layout-test failures in EWS

> On Dec 3, 2019, at 1:54 PM, Ryosuke Niwa wrote: > > > On Tue, Dec 3, 2019 at 9:29 AM Alexey Proskuryakov > wrote: > > Yes, I think that this makes more sense than retrying. > > What is the current behavior when a patch introduces substantial flakiness? > E.g. this

Re: [webkit-dev] Handling flaky layout-test failures in EWS

> What is the current behavior when a patch introduces substantial flakiness? For the patch which introduces substantial flakiness (and no consistent failure in EWS), prior to r253049 (which I landed two days back), the patch would be infinitely retried (unless atleast one of the flaky test fail

Re: [webkit-dev] Handling flaky layout-test failures in EWS

On Tue, Dec 3, 2019 at 9:29 AM Alexey Proskuryakov wrote: > > Yes, I think that this makes more sense than retrying. > > What is the current behavior when a patch introduces substantial > flakiness? E.g. this scenario: > > - First test run produces 5 failures. > - Second test run produces 5 diffe

Re: [webkit-dev] Handling flaky layout-test failures in EWS

Yes, I think that this makes more sense than retrying. What is the current behavior when a patch introduces substantial flakiness? E.g. this scenario: - First test run produces 5 failures. - Second test run produces 5 different failures. - Clean re-run produces no failures. This looks like dec

[webkit-dev] Handling flaky layout-test failures in EWS

Hi Everyone, We have various layout-tests which are flaky (which sometimes pass and sometimes fail/crash/timeout). EWS needs to work despite these flaky tests, and need to be able to tell whether the patch being tested introduced any test failure or not. In EWS, we have logic (same logic in bo