Zac - I didn't mean to offend! I got into this line of work by doing a lot of theoretical math, so if I use proof-by-exhaustion to point out an issue I have with your proposal, it doesn't mean that I don't want your help making firefox os a quality product.
More test coverage is always good. I also know that b2g developers haven't always been great about writing tests. For feature/bug code that landed without tests, late is much better than never. However, I think I made some valid points about why (for future work/bugs/patches) we should strive to either have one person write the feature/bugfix and the tests or have dev/qa work land in the same pull request/patch. It's also true (and I think you sensed from my response) that I want developers to write and maintain tests. I believe this is the only scalable way forward. That doesn't mean that, for the test coverage we already lack, we couldn't use some help. On Tue, Nov 19, 2013 at 5:50 PM, Zac Campbell <zcampb...@mozilla.com> wrote: > Well geez i was just offering, kindly I thought, to have QA write some > additional test coverage, especially against real device hardware. I > thought it would be valuable. > > But I can read between the lines; you want Python and QA's tests to sod > off. > > > > ----- Original Message ----- > From: Gareth Aye <gareth....@gmail.com> > To: Zac Campbell <zcampb...@mozilla.com> > Cc: Jonas Sicking <jo...@sicking.cc>, dev-g...@lists.mozilla.org, dev-b2g > <dev-b2g@lists.mozilla.org> > Sent: Tue, 19 Nov 2013 14:28:43 -0800 (PST) > Subject: Re: [b2g] Let QA write your Gaia/Marionette UI tests! > > Ok well that's all well and good, but I think we should be careful about > how we do this. Will there be separate pull requests to Gaia? I posit that > the correct way to develop is to review and land test code alongside > feature/bug code. Let's prove this to ourselves by exploring other, wrong > ways in which we can do this. > > Case 1: Suppose we review/land our tests before the feature/bug code. > > - This breaks the build. > - It's possible that we learn things while writing the feature/bug code > about the constraints of the problem and that the feature changes. In this > case, our tests have gotten old and crufty before they ever passed! > - It makes our generous reviewers take two steps out of their busy days > instead of one. > > Case 2: Suppose we review/land our tests after the feature/bug code. > > - We check untested code into source control. Do we even know if it works? > - If the feature/bug code gets backed out and the test stays, then we break > the build. > - We waste our reviewer's time again making them review two patches. > > Our proof is finished since the only options are to land our test code > before, alongside, or after our bug code. Therefore, if we're going to have > people other than feature/bug code writers developing automated test cases, > then everyone who's working on the bug must work together to submit a > single patch and/or pull request to Gaia. This workflow isn't what I've > observed so far with the Python test code, and I hope that we keep to it > for the future health of our project. > > Personally this seems like a lot of hassle to me (which is why I prefer to > write my own tests), but I am happy for others to do this so long as they > do it thoughtfully (in a way that doesn't waste reviewer time, stop good > tests from being written, and break the build). > > > > On Tue, Nov 19, 2013 at 3:54 AM, Zac Campbell <zcampb...@mozilla.com> > wrote: > > > Well I was trying to offer you a QA/automation team to do that for you! > > > > but I am interested in new test cases for new functionality coming up > that > > QA haven't been let in on yet or if there's coverage that your team needs > > for on-device testing to use hardware that desktopb2g doesn't have access > > to. We can cover some space that Travis cannot. > > > > Julien has requested some for Messaging app that we did and Rudy also > > asked for a keyboard/TBPL test which we did last week. > > > > > > > > > > > > > > > > On 18/11/13 16:58, Gareth Aye wrote: > > > > I will let you, but only if you use the js tooling so that I can review > > and maintain the tests :) > > > > > > On Sat, Nov 16, 2013 at 3:47 PM, Jonas Sicking <jo...@sicking.cc> wrote: > > > >> On Nov 14, 2013 3:57 AM, "Zac Campbell" <zcampb...@mozilla.com> wrote: > >> > > >> > Are you tired? is the cat bothering you for its food? Forgotten to > shave > >> for the last 6 days? Don't have time to write your UI tests? > >> > > >> > Then let QA do it for you! > >> > >> Hahaha, I love it! Zac++ > >> > >> / Jonas > >> _______________________________________________ > >> dev-b2g mailing list > >> dev-b2g@lists.mozilla.org > >> https://lists.mozilla.org/listinfo/dev-b2g > >> > > > > > > > > -- > > Best, > > Gareth > > > > > > > > > -- > Best, > Gareth > > -- Best, Gareth _______________________________________________ dev-b2g mailing list dev-b2g@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-b2g
