I have done this sort of thing a number of times too (sometimes successfully, sometimes not), and it's been multiple hours each time. I also recall hearing from several other people who have tried to do this and ended up giving up. I love it when mach just works and runs exactly what I want, but even then it's usually just the first step -- next I need to restrict the set of tests run, or run under a debugger, or leave the browser open after the test, or strace something, or recompile and use a different binary, or whatever. It's good when mach expands to cover more of these things, but it seems like I always fall off of the prepared path at some point.
It helps when it prints out command lines, though often they're buried in a bunch of other output and/or must be run within some modified environment that isn't recorded or isn't obvious. mozharness scripts suffer from the same sorts of problems. They work in automation, but often aren't very usable on a developer's box (eg they'll insist on downloading binaries and test packages instead of being able to run on a local checkout or binary.) And even when they work, you still end up needing to dig out the actual commands run to repeat them. Paths that are tested via automation work. Anything else is the wild wild west. It would be nice to have a developer-friendly path tested in automation, even if it's not the way the "real" jobs work. On 08/20/2014 10:22 AM, L. David Baron wrote: > On Wednesday 2014-08-20 09:14 +0100, Neil wrote: >> Gregory Szorc wrote: >> >>> Well, mach seems to be working for people doing m-c development. >> [Still needs a working build environment, while python runtests.py >> just used to need an objdir.] > I ran into this problem not long ago when I needed to run mochitests > on a different machine from the machine I was building on, because I > needed to run the mochitests in a configuration that supported OMTC. > (OMTC is not supported over VNC, so I couldn't run them on the > machine I was building on.) > > I ended up repeatedly using the command: > > /full/path/to/build/obj/firefox-debugopt/_virtualenv/bin/python > ./obj/firefox-debugopt/_tests/testing/mochitest/runtests.py --timeout 1000000 > --autorun --close-when-done --console-level=INFO > --testing-modules-dir=/full/path/to/build/obj/firefox-debugopt/_tests/modules > --extra-profile-file=obj/firefox-debugopt/dist/plugins > --test-path=layout/style/test/test_animations_omta_start.html > --setpref=layers.offmainthreadcomposition.async-animations=true > --setpref=layers.acceleration.force-enabled=true > > after rsyncing the objdir using the following rsync filter file: > > + /dist/ > + /dist/bin/ > H /dist/bin/core > H /dist/bin/core.* > H /dist/bin/test_* > H /dist/bin/Test* > H /dist/bin/*Test > H /dist/bin/*_unittest > H /dist/bin/*_unittests > H /dist/bin/*-tests > + /dist/plugins/ > H /dist/* > + /_tests/ > H /_tests/testing/mochitest/core > - /_virtualenv/ > + /mozinfo.json > + /config.status > H /* > > (which took me a few hours to figure out, if I recall). > > Doing the equivalent with reftest is substantially easier. > > > (I estimated that it was going to be faster to get that working than > to try to figure out how to use the packaged tests, since it was > possible to reverse-engineer from mochitest run inside mach, though > if there had been instructions on how to use packaged tests that > somebody had actually used before I'd likely have gone the other > way.) > > > This is part of why I dislike more and more layers being added to > the way in which we run tests; development often requires running > things in different-from-normal ways, and the more layers we have, > the harder that gets. > > -David > > > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
