On Fri, Mar 13, 2015 at 2:25 PM, L. David Baron <dba...@dbaron.org> wrote:
> On Friday 2015-03-13 13:02 -0700, Gregory Szorc wrote: > > FWIW, I don't think the documentation around the mach commands for test > > selection is that great. e.g. I'm not sure how many people realize that > > they can run `mach xpcshell-test test_foo.js` from the topsrcdir and all > > `test_foo.js` files under source control are executed. Perhaps we could > add > > some docs to `mach help` or drop some inline hints that people don't have > > to type so much. > > There are also some annoying inconsistencies between different mach > commands over what arguments they take. > > This is something that's more or less evolved organically as a result of having N different test runners, each with its own variation of options. It's been common for e.g. xpcshell tests to gain one nice option but not another. I was kinda hoping `mach test` could magically become "one command to run them all." However, there are still cases where an option only makes sense on one test type. So you are left with one command with a gazillion options or multiple commands with a syncing problem. Nobody wins. > > Something else that's quite annoying is that mach refuses to run > tests if certain aspects of my tree are out-of-date. Something I > commonly do is write a patch and a test, and then try to run the > test before compiling the patch to make sure that the test fails > without the patch. mach sometimes prevents me from doing this > because configure has been updated since I last built. > There is a tough balance to strike here. The mach testing commands need accurate knowledge of tests so they can run them properly. Test configs derive from moz.build evaluation which is dependent on configure being up to date. That's where the dependency comes from. If we didn't check the build system was up to date, you could be running tests with the wrong configuration state, which could result in tests running incorrectly or some tests not running at all! I don't quite follow how things don't work in your use case. Here's what I'd do: 1. Create a commit that introduces a new test 2. Test it 3. Create a commit that purportedly fixes the test 4. Build 5. Test and verify 6. Fold the commits When exactly is mach complaining about things not being up to date? Why isn't running tests against the state of your working copy what you intended? (I dare say that if this is what you want, you want an execution mode that is independent of source control state.) _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
