Re: Linting for common causes of oranges in mochitests (need ideas)

2017-07-07 Thread Gijs Kruitbosch
IME panels/popups/context-menus are flaky generally, and especially on Linux. I don't know how to meaningfully lint for this. The other thing is various incantations that try to do variations on the "load x in a tab" theme. It turns out that it's not trivial to not race with an expectant page

Re: Linting for common causes of oranges in mochitests (need ideas)

2017-07-07 Thread Andrew Halberstadt
On Fri, Jul 7, 2017 at 12:15 PM, Ehsan Akhgari wrote: > FWIW years ago I decided to act on trying to detect some of the patterns > indicated on that page automatically, see https://bugzilla.mozilla.org/s > how_bug.cgi?id=649012. It didn't use linting (because we didn't really > have any linting

Re: Linting for common causes of oranges in mochitests (need ideas)

2017-07-07 Thread Ehsan Akhgari
FWIW years ago I decided to act on trying to detect some of the patterns indicated on that page automatically, see https://bugzilla.mozilla.org/show_bug.cgi?id=649012. It didn't use linting (because we didn't really have any linting support back then) but the idea is the same, just with a diff

Re: Linting for common causes of oranges in mochitests (need ideas)

2017-07-06 Thread Andrew Halberstadt
Great idea, I'll add that to the list! Times actually are on that page too, I just didn't know about performance.now() as an alternative. Also, it was pointed out to me that in order to answer question 1) it would help to know how to disable these rules. Eslint rules can be disabled at the line or

Re: Linting for common causes of oranges in mochitests (need ideas)

2017-07-06 Thread Chris Peterson
On 7/6/17 11:47 AM, Andrew Halberstadt wrote: # Are there additional things not listed on that page that we could lint for? Do we want to discourage tests from using Date (`new Date` or `Date.now()`) for measuring time? Dates are affected by time zones, DST, and clock skew issues jumping fo

Linting for common causes of oranges in mochitests (need ideas)

2017-07-06 Thread Andrew Halberstadt
I'm looking into creating custom eslint rules that aim to avoid common causes of oranges in tests. We have an MDN page containing some of these already. Some of those patterns might be pretty hard to catch with a li