You can now add "--tag" arguments to try syntax and they will get passed to test harnesses in your try push. Details of the implementation are in bug 978846, but if you're interested in passing other arguments from try syntax to a test harness, this can be done by adding those arguments to testing/config/mozharness/try_arguments.py. Note this is still rather coarse in the sense that arguments are forwarded without regard for whether a harness supports a particular argument, but I can imagine it being useful in a number of cases (for instance, when testing the feature with xpcshell and "--tag devtools", I was able to get feedback in about ten minutes whether things were working rather than waiting for every xpcshell test to run).
Chris On Thu, Apr 2, 2015 at 2:22 PM, Andrew Halberstadt <ahalberst...@mozilla.com > wrote: > Minor update. It was pointed out that other list-like manifestparser > attributes (like head and support-files) are whitespace delimited instead > of comma delimited. To be consistent I switched tags to whitespace > delimitation as well. > > E.g both these forms are ok: > > [test_foo.html] > tags = foo bar baz > > [test_bar.html] > tags = > foo > bar > baz > > -Andrew > > > On 31/03/15 12:30 PM, Andrew Halberstadt wrote: > >> As of bug 987360, you can now run all tests with a given tag for >> mochitest (and variants), xpcshell and marionette based harnesses. Tags >> can be applied to either individual tests, or the DEFAULT section in >> manifests. Tests can have multiple tags, in which case they should be >> comma delimited. To run all tests with a given tag, pass in --tag <tag >> name> to the mach command. >> >> For example, let's say we want to group all mochitest-plain tests >> related to canvas together. First we'd add a 'canvas' tag to the DEFAULT >> section in >> >> https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/mochitest.ini >> >> >> [DEFAULT] >> tags = canvas >> >> We notice there is also a canvas related test under dom/media, namely: >> >> https://dxr.mozilla.org/mozilla-central/source/dom/media/test/mochitest.ini#541 >> >> >> Let's pretend it is already tagged with the 'media' tag, but that's ok, >> we can add a second tag no problem: >> >> [test_video_to_canvas.html] >> tags = media,canvas >> >> Repeat above for any other tests or manifests scattered in the tree that >> are related to canvas. Now we can run all mochitest-plain tests with: >> >> ./mach mochitest-plain --tag canvas >> >> You can also run the union of two tags by specifying --tag more than >> once (though the intersection of two tags is not supported): >> >> ./mach mochitest-plain --tag canvas --tag media >> >> So far the xpcshell (./mach xpcshell-test --tag name) and marionette >> (./mach marionette-test --tag name) commands are also supported. Reftest >> is not supported as it has its own special manifest format. >> >> Applying tags to tests will not affect automation or other people's >> tags. So each organization or team should feel free to use tags in >> whatever creative ways they see fit. Eventually, we'll start using tags >> as a foundation for some more advanced features and analysis. For >> example, we may implement a way to run all tests with a given tag across >> multiple different suites. >> >> If you have any questions or things aren't working, please let me know! >> >> Cheers, >> Andrew >> > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
