Can we set it up so we can manually runs tests on opt builds; but they aren't by default?
I've had many instances where opt (and pgo) fail; but I can't reproduce a test failure locally and can only do it on try. Letting me run that test on the opt build will save the additional pgo build time (both the cloud-cost time and the developer turn-around time.) -tom On Thu, Jan 3, 2019 at 4:20 PM jmaher <joel.ma...@gmail.com> wrote: > > I would like to propose that we do not run tests on linux64-opt, > windows7-opt, and windows10-opt. > > Why am I proposing this: > 1) All test regressions that were found on trunk are mostly on debug, and in > fewer cases on PGO. There are no unique regressions found in the last 6 > months (all the data I looked at) that are exclusive to OPT builds. > 2) On mozilla-beta, mozilla-release, and ESR, we only build/test PGO builds, > we do not run tests on plan OPT builds > 3) This will reduce the jobs (about 16%) we run which in turn reduces, cpu > time, money spent, turnaround time, intermittents, complexity of the > taskgraph. > 4) PGO builds are very similar to OPT builds, but we add flags to generate > profile data and small adjustments to build scripts behind MOZ_PGO flag > in-tree, then we launch the browser, collect data, and repack our binaries > for faster performance. > 5) We ship PGO builds, not OPT builds > > What are the risks associated with this? > 1) try server build times will increase as we will be testing on PGO instead > of OPT > 2) we could miss a regression that only shows up on OPT, but if we only ship > PGO and once we leave central we do not build OPT, this is a very low risk. > > I would like to hear any concerns you might have on this or other areas which > I have overlooked. Assuming there are no risks which block this, I would > like to have a decision by January 11th, and make the adjustments on January > 28th when Firefox 67 is on trunk. > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
