On Fri, Jan 18, 2019 at 10:36 PM Joel Maher <joel.ma...@gmail.com> wrote:
> Are there any concerns with this latest proposal? > This proposal sounds great to me. Thank you! Jan On Thu, Jan 17, 2019 at 12:52 PM Jan de Mooij <jdemo...@mozilla.com> wrote: > >> Hi Joel, >> >> Can you say more about this point in your original email: "3) This will >> reduce the jobs (about 16%) we run which in turn reduces, cpu time, money >> spent, turnaround time, intermittents, complexity of the taskgraph." It >> seems to me that if we remove non-PGO opt builds even on Try, we might use >> more cpu time because there are so many Try pushes requesting opt builds. >> Do we have data on this? >> >> Thanks, >> Jan >> >> On Thu, Jan 17, 2019 at 5:45 PM jmaher <joel.ma...@gmail.com> wrote: >> >>> Following up on this, thanks to Chris we have fast artifact builds for >>> PGO, so the time to develop and use try server is in parity with current >>> opt solutions for many cases (front end development, most bisection cases). >>> >>> I have also looked in depth at what the impact on the integration >>> branches would be. In the data set from July-December (H2 2018) there were >>> 11 instances of tests that we originally only scheduled in the OPT config >>> and we didn't have PGO or Debug test jobs to point out the regression (this >>> is due to scheduling choices). Worse case scenario is finding the >>> regression on PGO up to 1 hour later 11 times or roughly 2x/month. >>> Backfilling to find the offending patch as we do now 24% of the time would >>> be similar time. In fact running the OPT jobs on Debug instead would >>> result in same time for all 11 instances (due to more chunks on debug and >>> similar runtimes). In short, little to no impact. >>> >>> Lastly there was a pending question about talos. There is an edge case >>> where we can see a regression on talos that is PGO, but it is unrelated to >>> the code and just a side effect of how PGO works. I looked into that in >>> https://bugzilla.mozilla.org/show_bug.cgi?id=1514829. I found that if >>> we didn't get opt alerts that we would not have missed any regressions. >>> Furthermore, for the regressions, for the ones that were pgo only >>> regressions (very rare) there were many other regressions at the same time >>> (say a build change, or test change, etc.) and usually these were accepted >>> changes, backed out, or investigated on a different test or platform. In >>> the past when we have determined a regression is a PGO artifact we have >>> resolved it as WONTFIX and moved on. >>> >>> Given this summary, I feel that most concerns around removing testing >>> for OPT are addressed. I would also like to extend the proposal to remove >>> the OPT builds since no unit or perf tests would run on there. >>> >>> As my original timeline is not realistic, I would like to see if there >>> are comments until next Wednesday- January 23rd, then I can follow up on >>> remaining issues or work towards ensuring we start the process of making >>> this happen and what the right timeline is. >>> _______________________________________________ >>> dev-platform mailing list >>> dev-platform@lists.mozilla.org >>> https://lists.mozilla.org/listinfo/dev-platform >>> >> _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
