Re: Reorganization of Firefox-UI tests in mozilla-central

2017-10-31 Thread Martijn
Update MDN docs when moving Marionette tests all over the place. On Wed, Nov 9, 2016 at 11:16 AM, Henrik Skupin wrote: > Henrik Skupin wrote on 01/09/16 17:37: > > After a longer time without a response I would like to give a follow-up > on where we stand at the moment and what the future will be

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-11-09 Thread Henrik Skupin
Henrik Skupin wrote on 01/09/16 17:37: After a longer time without a response I would like to give a follow-up on where we stand at the moment and what the future will be for those tests. But first, I want to say thanks to everyone who participated in this thread. The received feedback was very h

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-05 Thread Henrik Skupin
L. David Baron wrote on 09/04/2016 10:02 AM: > Recently, sheriffing practices have changed so that intermittent > failure bugs that show up in different tests now have a separate > bugzilla bug for each test they occur in. This causes: > > 1. a large number of rarely-occurring bugs -- enough th

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-04 Thread L. David Baron
On Friday 2016-09-02 11:49 -0700, Wes Kocher wrote: > I don't believe that's a workable situation. At the moment, the policy is > that every new intermittent failure gets a bug filed for the purpose of > tracking it. There's talk (and has been for at least a year or two, now) that > work will be

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Matthew N.
On 2016-09-02 2:29 AM, Gijs Kruitbosch wrote: On 02/09/2016 00:15, Matthew N. wrote: On 2016-09-01 9:24 AM, Gijs Kruitbosch wrote: On 01/09/2016 16:37, Henrik Skupin wrote: Do those locations sound good? I have heard at least once that "firefox_ui" might not be the best choice as folder name,

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Andrew McCreight
On Fri, Sep 2, 2016 at 12:11 PM, Ryan VanderMeulen wrote: > On 9/2/2016 2:56 PM, Andrew McCreight wrote: > >> In DOM triage, we've just set up our list to not include bugs with the >> keyword intermittent failure. Ideally, we'd see intermittent failures that >> have a high rate, but given that sh

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Ryan VanderMeulen
On 9/2/2016 2:56 PM, Andrew McCreight wrote: In DOM triage, we've just set up our list to not include bugs with the keyword intermittent failure. Ideally, we'd see intermittent failures that have a high rate, but given that sheriffs keep an eye on oranges, they can look for somebody to fix freque

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Andrew McCreight
igher ups > for someone(s) to dedicate some time to work on it. > > KW > > From: e...@mozilla.com > Date: Fri, 2 Sep 2016 11:35:26 -0700 > Subject: Re: Reorganization of Firefox-UI tests in mozilla-central > To: m...@hskupin.info > CC: dev-platform@lists.mozilla.org; kwi

RE: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Wes Kocher
on it. KW From: e...@mozilla.com Date: Fri, 2 Sep 2016 11:35:26 -0700 Subject: Re: Reorganization of Firefox-UI tests in mozilla-central To: m...@hskupin.info CC: dev-platform@lists.mozilla.org; kwie...@gmail.com Henrik, It's not clear to me if this is just moving the location of existing tests,

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Emma Humphries
Henrik, It's not clear to me if this is just moving the location of existing tests, or if you're bringing in new tests. If it's the later, I'm concerned that we're going to get more intermittents filed in Bugzilla, which adds noise to the triage process. If you are adding more tests, then would

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Gijs Kruitbosch
On 02/09/2016 13:49, Henrik Skupin wrote: Gijs Kruitbosch wrote on 09/02/2016 11:37 AM: We should be doing the same for firefox-ui tests. Not just to avoid duplication of files in archives, but because otherwise, if we want to add new tests somewhere else, we both have to add the manifests to th

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Henrik Skupin
Gijs Kruitbosch wrote on 09/02/2016 01:22 PM: > AIUI, if we can run tests directly from a checkout, we no longer need > the common.tests.zip archive to exist (or at least not to have the tests > in them), so the problem is moot, no? Those files are used by test machines for all of our test suite

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Henrik Skupin
Gijs Kruitbosch wrote on 09/02/2016 01:22 PM: > AIUI, if we can run tests directly from a checkout, we no longer need > the common.tests.zip archive to exist (or at least not to have the tests > in them), so the problem is moot, no? I think so. There should not be a need to have to run "mach bu

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Henrik Skupin
Gijs Kruitbosch wrote on 09/02/2016 11:37 AM: > I am not familiar with this bit of our build architecture, but as far as > I can tell from a quick look it builds bits of the zipfile off the > objdir. So it collects mochitests from $OBJDIR/_tests/testing/mochitest, > where (again, AFAICT) things

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Gijs Kruitbosch
On 02/09/2016 10:54, James Graham wrote: On 02/09/16 10:37, Gijs Kruitbosch wrote: On 02/09/2016 08:08, Henrik Skupin wrote: The problematic piece here will be the package-tests step which currently picks complete subfolders. It would mean if we mix-up tests for firefox-ui-tests and eg. mochi

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread James Graham
On 02/09/16 10:37, Gijs Kruitbosch wrote: On 02/09/2016 08:08, Henrik Skupin wrote: The problematic piece here will be the package-tests step which currently picks complete subfolders. It would mean if we mix-up tests for firefox-ui-tests and eg. mochitests all would end-up twice in the common

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Gijs Kruitbosch
On 02/09/2016 08:08, Henrik Skupin wrote: Gijs Kruitbosch wrote on 09/01/2016 06:24 PM: As I did over IRC, I would like to strongly object to the continued use of per-test-type subfolders in our test directories. You can already use a specific mach command per test type, and the tests are liste

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Gijs Kruitbosch
On 02/09/2016 00:15, Matthew N. wrote: On 2016-09-01 9:24 AM, Gijs Kruitbosch wrote: On 01/09/2016 16:37, Henrik Skupin wrote: Do those locations sound good? I have heard at least once that "firefox_ui" might not be the best choice as folder name, but that's how the harness is called, and corre

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Henrik Skupin
Matthew N. wrote on 09/02/2016 01:15 AM: >>> /browser/components/sessionstore/test/firefox_ui/ >>> * update tests:/toolkit/mozapps/update/tests/firefox_ui/ > > Is there a plan to merge those with > /toolkit/mozapps/update/tests/marionette? It seems unusual to use both > when this ne

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-02 Thread Henrik Skupin
Gijs Kruitbosch wrote on 09/01/2016 06:24 PM: > As I did over IRC, I would like to strongly object to the continued use > of per-test-type subfolders in our test directories. You can already use > a specific mach command per test type, and the tests are listed in > different manifests, *and* th

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-01 Thread Matthew N.
On 2016-09-01 9:24 AM, Gijs Kruitbosch wrote: (CC'ing firefox-dev which doesn't seem to have had the original email though it should have done, please follow up to m.d.platform.) On 01/09/2016 16:37, Henrik Skupin wrote: * locationbar tests:/browser/base/content/test/urlbar/firefox_ui/

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-01 Thread Andreas Tolfsen
Henrik Skupin writes: > Do those locations sound good? I have heard at least once that > "firefox_ui" might not be the best choice as folder name, but that's > how the harness is called, and corresponds to what we have for other > harnesses too. I would suggest s/firefox_ui/ui/g, or to eliminate

Re: Reorganization of Firefox-UI tests in mozilla-central

2016-09-01 Thread Gijs Kruitbosch
(CC'ing firefox-dev which doesn't seem to have had the original email though it should have done, please follow up to m.d.platform.) On 01/09/2016 16:37, Henrik Skupin wrote: * locationbar tests:/browser/base/content/test/urlbar/firefox_ui/ * private browsing tests: /browser/compone

Reorganization of Firefox-UI tests in mozilla-central

2016-09-01 Thread Henrik Skupin
Hello, Via bug 1272145 we want to move our existing Firefox-UI tests from /testing/firefox-ui/ closer to the code which these are testing, so that the former location only contains the test harness and appropriate unit tests. Link to the current set of tests: https://dxr.mozilla.org/mozilla-centr