I wonder if this isn't (in large part) a design problem disguised as a behavior problem. The existing try syntax (even with try chooser) is so finicky and filled with abbreviations that even after years of working with it, I still regularly have to look up stuff and sometimes when I've been in a hurry, I've done something more general than I really needed because it was just too painful to figure out the exact thing.
I'd be pretty surprised if developers newer to the mozilla infrastructure than I didn't end up doing this sort of thing substantially more frequently. https://ahal.ca/blog/2017/mach-try-fuzzy/ seems like a fine step in the right direction, and maybe that'll be enough. But I do wonder if the path to saving substantial time and money in the long run is to invest some real user-research / UX / design time into designing a try configurator where it requires effort to do the unnecessarily expensive thing, as opposed to the current situation, where it requires effort to avoid the expensive thing. Dan 2017-09-15 9:46 GMT-07:00 Geoffrey Brown <gbr...@mozilla.com>: > Masayuki, your try push had trouble because you requested > "mochitest-2" instead of "mochitest-e10s-2". Non-e10s mochitests only > run on Android and Windows now. You probably wanted something like: > > https://treeherder.mozilla.org/#/jobs?repo=try&revision= > d68382f17d63f0674c62acc7242a9e406793895f > > > This is a good example of how a small deviation from "correct" try > syntax can have unexpected and frustrating consequences. > > - Geoff > > On Thu, Sep 14, 2017 at 7:15 PM, Masayuki Nakano <mnak...@mozilla.com> > wrote: > > I tried to say different point. See the treehearder log, mochitests > didn't > > run except on Win7 Debug, Android 4.3 API16+ Opt/Debug. So, try syntax > > parser or something is really broken. I often meet this kind of bug. > > > > > > On 9/15/2017 10:07 AM, Kris Maglione wrote: > >> > >> Your best bet is probably to use `mach try` with a specific set of test > >> directories. It will generate a set of --try-test-paths flags to > restrict > >> tests to those paths, and only run the first chunk of any group. Without > >> that, groups shift around too much to be reliable. > >> > >> On Fri, Sep 15, 2017 at 10:03:00AM +0900, Masayuki Nakano wrote: > >>> > >>> Even when I got the chunk numbers, specifying chunk numbers of > mochitests > >>> wouldn't work, see this log: > >>> > >>> https://treeherder.mozilla.org/#/jobs?repo=try&revision= > c09c7046ed0664e89f7224e1de5219c39c94c948 > >>> After that, I needed to rerun mochitests with |-u mochitests|. IIRC, I > >>> tried to kick the specific chunks with "Add new jobs", but didn't work. > >>> And also, when I try to investigate random oranges which are not > >>> reproducible on my environments, I want an option like > |--run-until-failure| > >>> and |--repeat REPEAT| in the try syntax. Because of no such options, I > need > >>> to trigger a lot of jobs manually and that may/might cause too many > oranges. > >>> > >>> On 9/15/2017 1:21 AM, Kyle Lahnakoski wrote: > >>>> > >>>> > >>>> You can try ActiveData, which stores all test results from the past > few > >>>> weeks. Here is an example query that shows the chunk number for each > >>>> run/build combo in the past day. ActiveData is sometimes more than a > >>>> day behind > >>>> > >>>> https://activedata.allizom.org/tools/query.html#query_id=4HHuBgDu > >>>> > >>>> { > >>>> "from":"unittest", > >>>> "select":[ > >>>> {"aggregate":"count"}, > >>>> {"value":"action.start_time","aggregate":"max"} > >>>> ], > >>>> "groupby":[ > >>>> "run.suite", > >>>> "run.chunk", > >>>> "result.test", > >>>> "build.platform", > >>>> "build.type", > >>>> "run.type" > >>>> ], > >>>> "where":{"and":[ > >>>> {"eq":{"build.branch":"mozilla-inbound"}}, > >>>> {"prefix":{"run.suite":"moch"}}, > >>>> {"gt":{"action.start_time":{"date":"today-day"}}}, > >>>> {"regex":{"result.test":".*browser_623779.js.*"}} > >>>> ]}, > >>>> "limit":1000 > >>>> } > >>>> > >>>> > >>>> > >>>> On 2017-09-14 11:49, Michael de Boer wrote: > >>>>>> > >>>>>> On 14 Sep 2017, at 17:48, Marco Bonardo <mbona...@mozilla.com> > wrote: > >>>>>> > >>>>>> When I need to retrigger a mochitest-browser test multiple times (to > >>>>>> investigate an intermittent), often I end up running all the > >>>>>> mochitest-browser tests, looking at every log until I find the chunk > >>>>>> where the test is, and retrigger just that chunk. The chunk number > >>>>>> changes based on the platform and debug/opt, so it's painful. > >>>>>> Is there a way to trigger only the chunk that will contain a given > >>>>>> test, so I can save running all of the other chunks? > >>>>> > >>>>> This! This! This! I’d love to be able to do this - would making > testing > >>>>> possible test failure fixes sooo much easier. > >>>>> > >>>>> Cheers, > >>>>> > >>>>> Mike. > >>>>> > >>>> > >>> > >>> > >>> -- > >>> Masayuki Nakano <mnak...@mozilla.com> > >>> Software Engineer, Mozilla > >> > >> > > > > > > -- > > Masayuki Nakano <mnak...@mozilla.com> > > Software Engineer, Mozilla > > > > -- > > You received this message because you are subscribed to the Google Groups > > "firefox-ci" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to firefox-ci+unsubscr...@mozilla.com. > > To post to this group, send email to firefox...@mozilla.com. > > To view this discussion on the web visit > > https://groups.google.com/a/mozilla.com/d/msgid/firefox- > ci/866a0e06-fbd9-c99b-451e-e20f80a12759%40mozilla.com. > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
