Wait - you're telling me that it is now possible to limit try pushes but not just jobs but tests within jobs?! Stop the presses: this is huge! If used by the masses, this could drastically reduce try turnaround times and decrease automation load and costs.
Could we encourage use of --tag by having the automation scheduler up-weight jobs that opt in to reduced load? On Thu, Apr 30, 2015 at 4:21 PM, Christopher Manchester < chmanches...@gmail.com> wrote: > You can now add "--tag" arguments to try syntax and they will get passed to > test harnesses in your try push. Details of the implementation are in bug > 978846, but if you're interested in passing other arguments from try syntax > to a test harness, this can be done by adding those arguments to > testing/config/mozharness/try_arguments.py. Note this is still rather > coarse in the sense that arguments are forwarded without regard for whether > a harness supports a particular argument, but I can imagine it being useful > in a number of cases (for instance, when testing the feature with xpcshell > and "--tag devtools", I was able to get feedback in about ten minutes > whether things were working rather than waiting for every xpcshell test to > run). > > Chris > > On Thu, Apr 2, 2015 at 2:22 PM, Andrew Halberstadt < > ahalberst...@mozilla.com > > wrote: > > > Minor update. It was pointed out that other list-like manifestparser > > attributes (like head and support-files) are whitespace delimited instead > > of comma delimited. To be consistent I switched tags to whitespace > > delimitation as well. > > > > E.g both these forms are ok: > > > > [test_foo.html] > > tags = foo bar baz > > > > [test_bar.html] > > tags = > > foo > > bar > > baz > > > > -Andrew > > > > > > On 31/03/15 12:30 PM, Andrew Halberstadt wrote: > > > >> As of bug 987360, you can now run all tests with a given tag for > >> mochitest (and variants), xpcshell and marionette based harnesses. Tags > >> can be applied to either individual tests, or the DEFAULT section in > >> manifests. Tests can have multiple tags, in which case they should be > >> comma delimited. To run all tests with a given tag, pass in --tag <tag > >> name> to the mach command. > >> > >> For example, let's say we want to group all mochitest-plain tests > >> related to canvas together. First we'd add a 'canvas' tag to the DEFAULT > >> section in > >> > >> > https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/mochitest.ini > >> > >> > >> [DEFAULT] > >> tags = canvas > >> > >> We notice there is also a canvas related test under dom/media, namely: > >> > >> > https://dxr.mozilla.org/mozilla-central/source/dom/media/test/mochitest.ini#541 > >> > >> > >> Let's pretend it is already tagged with the 'media' tag, but that's ok, > >> we can add a second tag no problem: > >> > >> [test_video_to_canvas.html] > >> tags = media,canvas > >> > >> Repeat above for any other tests or manifests scattered in the tree that > >> are related to canvas. Now we can run all mochitest-plain tests with: > >> > >> ./mach mochitest-plain --tag canvas > >> > >> You can also run the union of two tags by specifying --tag more than > >> once (though the intersection of two tags is not supported): > >> > >> ./mach mochitest-plain --tag canvas --tag media > >> > >> So far the xpcshell (./mach xpcshell-test --tag name) and marionette > >> (./mach marionette-test --tag name) commands are also supported. Reftest > >> is not supported as it has its own special manifest format. > >> > >> Applying tags to tests will not affect automation or other people's > >> tags. So each organization or team should feel free to use tags in > >> whatever creative ways they see fit. Eventually, we'll start using tags > >> as a foundation for some more advanced features and analysis. For > >> example, we may implement a way to run all tests with a given tag across > >> multiple different suites. > >> > >> If you have any questions or things aren't working, please let me know! > >> > >> Cheers, > >> Andrew > >> > > > > _______________________________________________ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
